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Hurricane Katrina in 2005 highlighted 
gaps in the nation's preparedness to 
respond effectively to catastrophic 
incidents. The Department of 
Homeland Security's (DHS) Federal 
Emergency Management Agency 
(FEMA) designed the Task Force for 
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program to advance and integrate 
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Defense (DOD), ran from September 
2008 to September 2010, and FEMA 
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states — Hawaii, Massachusetts, South 
Carolina, Washington, and West 
Virginia— with $350,000 each to 
develop plans, build relationships 
with stakeholders, and document 
lessons learned (i.e., TFER's stated 
objectives). As requested, GAO 
evaluated the extent to which (1) 
FEMA followed sound management 
practices in designing, administering, 
and evaluating TFER and (2) the five 
participating states satisfied TFER's 
stated objectives. GAO analyzed 
FEMA guidance, such as the TFER 
Pilot Information Package, conducted 
site visits to all five participant states, 
and met with relevant FEMA and 
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implementation of it. 
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What GAO Found 

FEMA developed program objectives and procedures for administering the 
TFER pilot, but did not develop other elements of sound management 
practices in designing, administering, and evaluating pilot programs that GAO 
identified from its prior work and social science literature. FEMA developed 
objectives for the pilot, but did not document standards for determining the 
pilot's success. FEMA also provided resources such as funding, training, and 
support, but FEMA did not always follow the procedures it established for 
TFER. For example, FEMA did not consistently conduct biweekly conference 
calls with the states, and four states reported that it would have been helpful 
if FEMA provided more guidance. FEMA did not develop a data analysis plan, 
which could have better ensured FEMA collected data on the extent to which 
the pilot states coordinated with key stakeholders and provided reasonable 
assurance that FEMA conducted a systematic assessment of TFER using 
comparable data across the five pilot states. In Spring 2010, FEMA announced 
TFER would not continue past the pilot stage before evaluating TFER on its 
merits in strengthening and advancing state catastrophic planning, but FEMA 
allowed the states to draw down the remaining TFER funds and continue to 
complete initiatives started under TFER through August 2011. GAO previously 
reported in April 2009 that FEMA faced challenges in assessing pilot program 
data, and FEMA officials reported the agency does not have pilot program 
policy guidance. In the absence of this, FEMA lacks a systematic approach to 
developing, administering, and evaluating pilot programs. FEMA could better 
ensure other pilot programs meet their intended goals by developing policies 
and guidance that include sound management practices. 

All five states have taken steps to follow FEMA's guidance to address TFER's 
objectives, but no state has fully addressed them all (see table below). First, 
two of the five states have completed draft catastrophic plans, and all five 
states reported following FEMA's planning process. Second, all five states 
built relationships with stakeholders such as state agencies and FEMA, but 
state officials said coordination with DOD — a key federal stakeholder who 
may be called upon to assist in disaster response — was limited. State officials 
reported not coordinating with DOD because they did not have draft plans for 
DOD officials to review. Third, all states have documented lessons learned to 
date, but four states have not exercised TFER plans to determine their 
effectiveness in the event of an emergency. 

Progress in Meeting FEMA's TFER Pilot Program Objectives 
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• Substantial progress: average state progress was greater than 80 percent 
© Some progress: average state progress ranged from 20 percent to 80 percent 
O Little to no progress: average state progress was less than 20 percent 

Source: GAO analysis of state information. 
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GAP 

Accountability * Integrity * Reliability 



United States Government Accountability Office 
Washington, DC 20548 



April 8, 2011 

Congressional Requesters 

Hurricane Katrina in 2005 highlighted gaps in the nation's preparedness to 
respond effectively to catastrophic incidents. In the aftermath of 
Hurricane Katrina, the Federal Emergency Management Agency (FEMA), a 
component of the Department of Homeland Security (DHS), was charged 
with leading the nation in preparing for disasters in coordination with 
other federal entities like the Departments of Defense (DOD) and Justice; 
the 50 states, the District of Columbia, and five territories; tribal, city, and 
county governments; nonprofit organizations; and private entities. 1 The 
Department of Homeland Security Appropriations Act, 2010 appropriated 
approximately $7. 1 billion to FEMA, and FEMA has allocated more than 
$1.8 billion to preparedness grant programs. 2 

The Task Force for Emergency Readiness (TFER) pilot program was 
launched by FEMA in September 2008 to strengthen state preparedness for 
catastrophic disasters by facilitating greater capacity in and more 
comprehensive integration of planning efforts across all levels of 
government through $1.75 million in grant funds. In particular, TFER 
emphasized integration of planning efforts across sectors, jurisdictions, 
and functional disciplines, as well as integration among state, regional, and 
federal agencies, primarily FEMA and DOD. The TFER grant funds 
allowed states to hire three full-time planners with civilian and military 
planning expertise to develop catastrophic plans. FEMA selected five 
states to participate in this pilot program: Hawaii, Massachusetts, South 
Carolina, Washington, and West Virginia. 



1 The Post-Katrina Emergency Management Reform Act of 2006 provided that FEMA must 
lead the nation's efforts to prepare for, protect against, recover from, and mitigate against 
the risk of natural disasters, acts of terrorism, and other man-made disasters, including 
catastrophic incidents. 6 U.S.C. § 313(b)(2)(A). 

2 Pub. L. No. 111-83, 123 Stat. 2142 (2009). 
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Planning is a key component of national preparedness. 3 According to 
FEMA planning guidance, planning provides a methodology to examine 
the entire life cycle of a potential crisis, determine required capabilities, 
and help stakeholders learn and practice their roles for building and 
sustaining national preparedness capabilities against terrorist attacks and 
other hazards. This is particularly important in a catastrophic incident, 
defined as any natural disaster, act of terrorism, or man-made disaster that 
results in extraordinary levels of casualties or damage or disruption 
severely affecting the population, infrastructure, environment, economy, 
national morale, or government functions in an area. 4 Catastrophic events 
differ from a normal disaster in the severity of the damage, number of 
persons affected, and the scale of preparation and response required. They 
quickly overwhelm or incapacitate local and/or state response capabilities, 
thus requiring coordinated assistance from outside the affected area. 

You asked us to review the TFER pilot program. Our objectives for this 
report were to identify the extent to which (1) FEMA followed sound 
management practices in developing, administering, and evaluating the 
TFER pilot program and (2) the five participating states satisfied the TFER 
pilot program's stated objectives. 

To conduct this work, we completed site visits to all five participant states 
from June through September 2010 and interviewed relevant FEMA, DOD, 
and DHS headquarters and regional officials. During our site visits, we met 
with the planners hired through the TFER pilot program and other state, 
local, and DOD and FEMA regional officials to discuss FEMA's 
management of the TFER pilot program and the states' implementation of 
it. 

To identify the extent to which FEMA followed sound management 
practices in developing, administering, and evaluating TFER, we analyzed 
program guidance and other key documents including the TFER Pilot 
Information Package and the TFER Guidance and Application Kit. We also 



3 See, GAO, National Preparedness: FEMA Has Made Progress, but Needs to Complete and 
Integrate Planning, Exercise, and Assessment Efforts, GAO-09-369 (Washington, D.C.: 
Apr. 30, 2009); Emergency Preparedness: FEMA Faces Challenges Integrating 
Community Preparedness Programs into Its Strategic Approach GAO-10-193, 
(Washington, D.C.: Jan. 29, 2010). 

4 As defined in the Post-Katrina Act. 6 U.S.C. § 701(4). 
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analyzed our prior work on pilot programs and program management 5 as 
well as social science literature 6 to determine elements of sound 
management practices. We grouped these elements into three categories: 
design, administration, and evaluation. We analyzed the program guidance, 
other documents, and information obtained from our interviews and site 
visits and compared the results of our work with the identified elements of 
sound management practices. Furthermore, we analyzed our prior work 
and DHS Office of Inspector General reports on FEMA grants 
administration and catastrophic preparedness to identify any past 
challenges. 7 

To identify the extent to which the five states participating in TFER 
addressed the program's stated objectives, we analyzed information that 
FEMA provided in the TFER Pilot Information Package and further 
defined in its TFER Evaluation Criteria document. The three program 
objectives included developing plans, building relationships, and 
documenting lessons learned, and FEMA provided criteria to measure 
progress towards addressing each of these objectives. We used FEMA's 
criteria to then assess states' progress toward addressing the three 
program objectives. Additionally, FEMA identified key components of 
each objective. To assess the states' progress towards addressing TFER's 
program objectives, we assigned a numerical ranking to each objective's 
components and then averaged the components' numerical rankings for 
each of the objectives. We then assigned designations to these averages. 8 
We determined that states could not receive a designation of "substantial 



5 GAO, Tax Administration: IRS Needs to Strengthen Its Approach for Evaluating the 
SRFMI Data-Sharing Pilot Program, GAO-09-45 (Washington, D.C.: Nov. 7, 2008); GAO, 
Designing Evaluations, GAO/PEMD-10.1.4 (Washington, D.C.: Mar. 1991). 

6 P.H. Rossi, M.W. Lipsey, and H.E. Freeman, Evaluation: A Systematic Approach 
(Thousand Oaks, Calif.: 2004); Project Management Institute, The Standard for Program 
Management® (Newton Square, Pa: 2006); and Daniel L. Stufflebeam, Program Evaluations 
Metaevaluation Checklist (Based on The Program Evaluation Standards), 1999. 

' GAO, National Preparedness: FEMA Has Made Progress, but Needs to Complete and 
Integrate Planning, Exercise, and Assessment Efforts, GAO-09-369 (Washington, D.C.: 
Apr. 30, 2009); GAO, Emergency Management: Observations on DHS's Preparedness for 
Catastrophic Disasters, GAO-08-868T (Washington, D.C.: June 11, 2008); Department of 
Homeland Security, Office of Inspector General, Efficacy of DHS Grant Programs, OIG-10- 
69 (Washington, D.C.: Mar. 22, 2010). 

8 Averages greater than 80 percent were designated as 'substantial progress'; averages from 
20 percent to 80 percent were designated as 'some progress'; and averages less than 20 
percent were designated as 'little or no progress.' 
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progress" for their objectives without meeting the criteria related to these 
key components. To assist in assessing states' progress, we reviewed and 
analyzed the documents obtained during our site visits that included draft 
plans, progress reports, and status briefings, and interviewed cognizant 
officials. As part of this analysis, we reviewed the guidance in FEMA's 
Comprehensive Preparedness Guide (CPG) 101, which provides 
instructions regarding plan structure, and we used this information to 
determine the extent to which the states' plans adhered to FEMA's 
guidance. 

We conducted this performance audit from May 2010 to April 2011 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the evidence 
obtained provides a reasonable basis for our findings and conclusions 
based on our audit objectives. Additional details on our scope and 
methodology are contained in appendix I. 



Background 



The National Response 
Framework Established a 
Tiered Response for 
Emergencies 



DHS issued the National Response Framework for federal, state, and local 
agencies to use in planning for emergencies. 9 It establishes standardized 
doctrine, terminology, and processes for responding to disasters and other 
catastrophic events in the United States. The framework is based on a 
tiered, graduated response; that is, incidents are managed at the lowest 
jurisdictional levels and supported by additional higher-tiered response 
capabilities as needed (see fig. 1). 



9 The National Response Plan preceded the Department of Homeland Security's National 
Response Framework (Washington, D.C.: January 2008). Several years earlier, the 
President issued Homeland Security Presidential Directive 8 — National Preparedness 
(Dec. 17, 2003) that called on the Secretary of Homeland Security to carry out and 
coordinate preparedness activities with public and private organizations involved in such 
activities. 
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Figure 1 : Roles of FEMA and DOD — the Primary Stakeholders for TFER — within the 
National Response Framework 



First level 
res ponders 




Local resources are exhausted 



Second level 
res ponders 



Local mutual aid resources 
are exhausted 



Third level 
res ponders 



State resources are exhausted 
(including the National Guard) 



Fourth level 
res ponders 



Interstate mutual aid 
resources are exhausted 



Governor requests assistance and 
a Presidential Declaration is issued 



Fifth level 
res ponders 



FEMA evaluates situation and 
coordinates federal response 



Federal government emergency 
support function resources are exhausted 



Seventh level 
res ponders 



FEMA requests DOD assistance 

First mission assignment is generally to activate 
the Defense Coordinating Officer and staff 



Source: GAO analysis of DOD, FEMA, and state information. 
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Note: Each Emergency Support Function comprises a coordinator, a primary agency, and support 
agencies — usually governmental agencies — that plan and support response activities. Support 
agencies are assigned based on their authorities, resources, and capabilities in a given functional 
area. According to the National Response Framework, if a state anticipates that its resources may be 
exceeded, the governor can request assistance from the federal government and/or other states 
through mutual aid and assistance agreements such as Emergency Management Assistance 
Compacts. 

Local governments respond to emergencies using their own resources or 
assistance from neighboring localities. For larger-scale incidents that 
overwhelm the capabilities of local governments, assistance from the state 
may be sought. Depending on the circumstances, states have capabilities, 
such as the National Guard, 10 that can help communities respond and 
recover. If additional resources beyond what an individual state can 
provide are required, the state may request assistance from other states 
through Emergency Management Assistance Compacts or from the federal 
government. 11 Catastrophic incidents, by definition, result in extraordinary 
levels of damage or disruption to government functions, and governors 
have the option to seek federal assistance. The federal government has a 
wide array of capabilities and resources to assist state and local agencies 
respond to incidents. In accordance with the National Response 
Framework and applicable laws including the Robert T. Stafford Disaster 
Relief and Emergency Assistance Act (Stafford Act), 12 various federal 
departments and agencies may play primary, coordinating, or supporting 
roles, based on their authorities and resources and the nature of the 
incident. In certain instances, national military capabilities may be 
requested to respond to an incident. Defense resources are committed 



10 The National Guard holds a unique dual status in that it performs its state missions under 
the command of the state's governor and federal missions as needed under the command 
of the President. Currently, the majority of the National Guard's personnel, training, and 
equipment are provided for its federal warfighting mission with funding appropriated to 
DOD. The National Guard can use capabilities provided by DOD when available to respond 
to domestic emergencies while operating under the command of the governors and 
generally paid for with state funding. Further, under certain circumstances such as 
homeland security-related activities or federally declared disasters, federal funding has 
been provided for missions carried out by the states' National Guard. 

11 Pub. L. No. 104-321, 110 Stat. 3877 (1996).The Emergency Management Assistance 
Compact is a congressionally ratified organization that provides form and structure to 
interstate mutual aid. Through this emergency management assistance compact, a 
disaster-impacted state can request and receive assistance from other member states 
quickly and efficiently, resolving two key issues upfront: liability and reimbursement. 

12 Pub. L. No. 100-707, 102 Stat. 4689 (1988) (codified as amended at 42 U.S.C. §§ 5121-5208). 
The Stafford Act is the most widely used authority under which federal disaster assistance 
occurs. 
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following approval by the Secretary of Defense or at the direction of the 
President. 

According to FEMA planning guidance, 13 plans must be integrated 
vertically among levels of government (e.g., local, state, and federal) to 
ensure a common operational focus and horizontally (e.g., health 
departments and law enforcement) to ensure that individual department 
and agency operations plans complement the jurisdiction's plans. This 
integration enables stakeholders to synchronize the sequence and scope of 
a planned response in terms of roles and responsibilities, place, and time 
to a catastrophic incident. Catastrophic incidents include those that are 
defined in the National Planning Scenarios, 14 which represent examples of 
the gravest dangers facing the United States, including terrorist attacks 
and natural disasters, and have been accorded the highest priority for 
federal planning efforts. 



FEMA and DOD Each Play FEMA's primary mission is to reduce the loss of life and property and 
a Role in Emergency protect the nation from all hazards, including natural disasters, acts of 

Response terrorism, and other man-made disasters, by leading and supporting the 

nation in a risk-based, comprehensive emergency management system of 
preparedness, protection, response, recovery, and mitigation. 15 As 
described in the National Response Framework, planning is one of six 
activities essential to preparing for an incident. Other activities include 
training; equipping; exercising; and evaluating and improving to build 
tribal, local, state, regional, and national capabilities necessary to respond 
to any type of disaster. 



13 In 2009, FEMA published CPG 101 that provides general guidelines on developing 
emergency operations plans. It promotes a common understanding of the fundamentals of 
planning and decision making to help operations planners examine a hazard or threat and 
produce integrated, coordinated, and synchronized plans. This guide helps emergency and 
homeland security managers in state, territorial, tribal, and local governments in their 
efforts to develop and maintain viable all-hazard emergency plans. FEMA recommends that 
organizations responsible for developing emergency operations plans use CPG 101 to guide 
their efforts. 

14 DHS's all-hazards National Planning Scenarios are an integral part of the department's 
capabilities-based approach to implementing Homeland Security Presidential Directive 8: 
National Preparedness. The scenarios serve as the foundation for the development of 
homeland security tasks, target capabilities, and standards and performance metrics. 

15 FEMA's mission is outlined in 6 U.S.C. § 313(b)(1). 
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As shown in figure 2, FEMA has 10 regional offices nationwide. Each 
region serves several states, and FEMA regional personnel work generally 
with the states to help plan for disasters, develop mitigation programs, and 
meet other needs when major disasters occur. Regional office locations 
are starred in the figure below, and in some cases are located within a 
pilot state. 



Figure 2: Location of TFER Pilot States within FEMA Regions 



Alaska 




Source: FEMA; Map Resources (map). 



U.S. Northern Command (NORTHCOM) and U.S. Pacific Command 
(PACOM) are the combatant commands charged with carrying out DOD's 
domestic civil support mission, which involves responding to the needs of 
56 separate and often unique state and territorial governments. 
Additionally, NORTHCOM and PACOM coordinate with numerous federal 
agencies that also have roles in planning for and responding to a variety of 
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incidents in the homeland. As part of the lessons learned from Hurricane 
Katrina, both NORTHCOM and PACOM have established Defense 
Coordinating Officers with staff to serve as DOD representatives to civilian 
authorities in the 10 FEMA regions. 16 Defense Coordinating Officers are 
full-time senior-level military officers who provide liaison support and 
requirements validation, and they serve as single points of contact for 
state, tribal, local, and other federal authorities that need DOD support. In 
that way they are able to develop relationships with civilian authorities 
and gain an understanding of civilian capabilities so that DOD will know 
what, if anything, it may be called upon to provide in the event of a 
disaster or other incident. Additional defense resources include 
Emergency Preparedness Liaison Officers who are senior Reserve officers 
that are expected to assist the Defense Coordinating Officers in 
coordinating the provision of military personnel, equipment, and supplies 
to support the emergency relief and cleanup efforts of civilian 
authorities. 17 When NORTHCOM and PACOM are called upon to support 
civilian authorities, in most cases support will be localized, limited, and 
specific. When the scope of the disaster is reduced to the point where the 
primary federal agency can again assume full control and management 
without military assistance, NORTHCOM and PACOM will withdraw. 



16 NORTHCOM has designated 10 Defense Coordinating Officers with staff, one in each of 
the 10 FEMA regions. However, since FEMA Region IX is located in both NORTHCOM and 
PACOM, NORTHCOM has a Defense Coordinating Officer assigned to the FEMA regional 
office in California and PACOM has established two Defense Coordinating Officers of its 
own, one under the Army for the state of Hawaii and the territory of American Samoa, and 
one under the Navy for the territories of Guam and the Northern Marianas. Puerto Rico and 
the U.S. Virgin Islands are part of FEMA Region II and are covered by the NORTHCOM 
Defense Coordinating Officer for Region II. 

17 In March 2010, we reported that the ability of Defense Coordinating Officers and 
Emergency Preparedness Liaison Officers to coordinate and provide DOD capabilities to 
civil authorities may be limited because DOD has not delineated the roles, responsibilities, 
and command and control relationships between the Defense Coordinating Officers and 
Emergency Preparedness Liaison Officers. DOD officials told us that there is confusion 
between the military services and the Defense Coordinating Officers regarding the proper 
employment of Emergency Preparedness Liaison Officers. DOD concurred with the 
findings in our report. GAO, Homeland Defense: DOD Can Enhance Efforts to Identify 
Capabilities to Support Civil Authorities during Disasters, GAO-10-368 (Washington, 
D.C.: Mar. 30, 2010). 
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The TFER pilot program was first envisioned by senior DOD leadership to 
address the National Planning Scenarios through supporting and 
strengthening the catastrophic disaster preparedness of individual states. 
However, since DOD is always a supporting agency when responding to 
catastrophic events in the homeland, DOD leadership did not think it was 
appropriate for DOD to administer the pilot program, and FEMA 
administered and funded the pilot program. Although FEMA administered 
TFER, pilot program guidance stated that the TFER program was to place 
a special emphasis on coordination and integration between the pilot 
states, FEMA, and DOD. Further, a key measure of achievement in the 
pilot program is the degree to which DOD stakeholders are integrated as 
full partners in the development of states' plans. 

FEMA designed TFER to help assess, strengthen, and advance state 
catastrophic preparedness planning across state and local agencies and 
the private sector, as well as achieve a fuller integration of state planning 
efforts with federal agency partners through a deliberate blending of civil- 
military planning expertise. The TFER pilot program centered on the 
creation and employment of a dedicated planning team. Specifically, the 
planning team was to synchronize catastrophic planning efforts at the 
respective state, regional, and federal levels; prepare planning documents; 
identify gaps in capabilities; and assist in the overall organization, 
administration, and improvement of the states' catastrophic preparedness 
planning capacity. The three objectives of TFER are to (1) develop plans, 
(2) build relationships, and (3) document lessons learned. 

FEMA designed TFER to run 18 months, beginning on September 1, 2008, 
and ending on March 31, 2010. The pilot program was extended another 6 
months to September 30, 2010 — for a total time of 2 years — to allow states 
more time to accomplish their planning goals. As shown in figure 3 , FEMA 
announced in March 2010 that TFER would not continue past the pilot 
stage, but allowed the pilot states to draw down the remaining TFER funds 
and continue to complete initiatives started under TFER through August 
2011. 



DOD Envisioned TFER, 
but FEMA Implemented 
the Pilot Program 
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Figure 3: Timeline of TFER-related Activities 



September 
2008: FEMA 
releases TFER 
Guidance and 
Application Kit 



April 2009: 

Training held in 
West Virginia for 
TFER planners 



March 2010: 

• FEMA informs 
stakeholders 
TFER will not 
continue past the 
pilot stage 

• Original 

completion date of 
TFER; 6-month 
extension granted 



September 201 - August 201 1 : 

TFER's 6-month extension 
expires September 2010, 
however, FEMA allows states to 
complete planning activities 
initiated under TFER and finish 
drawing down funds until August 
2011" 



February 2009: 

Participating states begin 
to hire TFER planners 



June 201 - February 201 1 : 

FEMA conducts its evaluation 
of TFER 



Source: GAO analysis of FEMA data. 

"FEMA's grant extension to the TFER pilot states extended the period of performance for the grant, 
meaning states could continue to draw down TFER funding and work on catastrophic planning 
activities. 

FEMA limited the pilot program to five states, each of which received 
$200,000 in the first year of the pilot program, and an additional $150,000 
for the remainder of the program, for overall program funds totaling $1.75 
million. It selected applicants to achieve a representative cross-section of 
geographic differences, FEMA regions, threat of catastrophic events, state 
emergency management structures, and participation in a separate FEMA 
catastrophic planning grant. 18 As discussed in appendix II, FEMA provides 
funds to states and localities for catastrophic planning through other grant 
programs. For example, FEMA's State Homeland Security Program 
provides funding that may be used for planning efforts that enable states 



18 FEMA selected four states that receive funds through the Regional Catastrophic Planning 
Grant Program — Hawaii, Massachusetts, Washington, and West Virginia. South Carolina 
does not receive funds through this grant and served as a comparison state. The Regional 
Catastrophic Planning Grants Program is intended to enhance catastrophic incident 
preparedness in selected high-risk, high-consequence urban areas and their surrounding 
regions. 
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to prioritize needs, build capabilities, update preparedness strategies, 
allocate resources, and deliver preparedness programs across disciplines 
and levels of government. 

FEMA administers the pilot program within its National Preparedness 
Directorate. 19 This directorate is responsible for overseeing the 
coordination and development of strategies necessary to prepare for all 
hazards. As part of this mission, the National Preparedness Directorate is 
to provide policy and planning guidance, training and exercises, and 
technical assistance that builds prevention, protection, response, and 
recovery capabilities. 

In the aftermath of Hurricane Katrina, FEMA began to address the 
challenges revealed by its response. FEMA's 2006 Nationwide Plan Review 
highlighted the need for fundamental planning modernization and 
observed, among other things, that: catastrophic planning efforts were 
unsystematic and uneven; planning expertise was insufficient for 
catastrophic incidents; and collaboration requirements were not well- 
defined, fostering a tendency to plan internally. In addition, according to 
FEMA, the 2006 Nationwide Plan Review revealed that 95 percent of state 
and urban area participants cited the need for federally funded planning 
support and technical assistance. FEMA also reported that in 2007, states 
and urban areas ranked planning among their highest grant funding 
priorities. 20 The Department of Homeland Security's Office of Inspector 
General recently reported that while progress has been made, the National 
Preparedness Directorate needs more effective coordination with state, 
local, and tribal governments. 21 Our prior work on FEMA, and specifically 
the National Preparedness Directorate, has highlighted the need for 
strategic planning that identifies outcomes and performance measures for 



The National Preparedness Directorate also administers the Regional Catastrophic 
Planning Grants Program. FEMA's Grant Program Directorate is responsible for 
administering all of the other FEMA preparedness grant programs. 

20 In July 2010, FEMA issued an update to the Nationwide Plan Review in which it reported 
that the state of national planning had substantially improved (e.g., states and urban areas 
reported updating their basic plans and increased confidence in planning for catastrophic 
events) and much had been accomplished to address the recommendations and findings 
from the 2006 review. 

21 DHS, Office of Inspector General, OIG-10-123, FEMA's Preparedness for the Next 
Catastrophic Disaster - An Update (Washington, D.C., Sep. 27, 2010). 
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the Directorate's programs. 22 In October 2010, we also reported that since 
April 2009, FEMA has made limited progress in assessing preparedness 
capabilities and has not yet developed national preparedness capability 
requirements based on established metrics to provide a framework for 
these assessments. 23 We reported that until such a framework is in place, 
FEMA will not have a basis to operationalize and implement its conceptual 
approach for assessing local, state, and federal preparedness capabilities 
against capability requirements to identify gaps for prioritizing 
investments in national preparedness. FEMA responded that it has made 
much progress since 2009 in meeting its legislative requirements and 
highlighted some of its specific achievements, such as the establishment of 
a working group to help consolidate and streamline reporting 
requirements for state, tribal, and local stakeholders. 



See, GAO, National Preparedness: FEMA Has Made Progress, but Needs to Complete 
and Integrate Planning, Exercise, and Assessment Efforts, GAO-09-369 (Washington, 
D.C.: Apr. 30, 2009); Emergency Preparedness: FEMA Faces Challenges Integrating 
Community Preparedness Programs into Its Strategic Approach, GAO-10-193 
(Washington, D.C.: Jan. 29, 2010). 

23 See, GAO, FEMA Capabilities Assessments: FEMA Has Made Limited Progress in 
Efforts to Develop and Implement a System to Assess National Preparedness 
Capabilities, GAO-11-51R (Washington, D.C.: Oct. 29, 2010). 
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FEMA Implemented 
Some, but Not All 
Elements of Sound 
Management 
Practices in Its 
Administration of the 
TFER Pilot Program, 
and Does Not Have 
Policy Guidance on 
Developing Pilot 
Programs 



FEMA developed program objectives and a data collection plan — elements 
of sound management practices — but did not implement other elements of 
sound management practices, such as documenting standards for 
determining the program's success and did not always follow its stated 
processes and procedures for TFER. Further, FEMA does not have 
agencywide policy guidance for FEMA program managers to follow when 
developing pilot programs. In November 2008, we reported that pilot 
programs can more effectively inform future program rollout when sound 
management practices are followed. 24 Consistent with best practices in 
program management, 25 our guide for designing evaluations, 26 and our 
prior work, we identified sound management practices to design a pilot, to 
guide consistent implementation of a pilot, and to conduct analysis of the 
results. Our assessment of FEMA's implementation of sound management 
practices for TFER is shown in table 1. 



See, GAO, Tax Administration: IRS Needs to Strengthen Its Approach for Evaluating 
the SRFMI Data-Sharing Pilot Program, GAO-09-45 (Washington, D.C.: Nov. 7, 2008). 
Specifically, in GAO-09-45 we reported that a sound, well-developed and documented 
evaluation plan includes, at a minimum: (1) well-defined, clear, and measurable objectives; 
(2) criteria or standards for determining pilot-program performance; (3) clearly articulated 
methodology, including sound sampling methods, determination of appropriate sample size 
for the evaluation design, and a strategy for comparing the pilot results with other efforts; 
(4) a clear plan that details the type and source of data necessary to evaluate the pilot, 
methods for data collection, and the timing and frequency of data collection; and (5) a data 
analysis plan to track the program's performance and evaluate the final results of the 
project. 

25 See, for example, P.H. Rossi, M.W. Lipsey, and H.E. Freeman, Evaluation: A Systematic 
Approach (Thousand Oaks, Calif.: 2004); and the Project Management Institute. The 
Standard for Program Management® (Newton Square, Pa: 2006). Specifically, 
Evaluation: A Systematic Approach covers evaluation research activities used in 
appraising the design, implementation, effectiveness, and efficiency of social programs. 
Also, The Standard for Program Management describes program phases to facilitate 
program governance, enhanced control, and coordination of program and project 
resources and overall risk management. 

26 See, GAO, Designing Evaluations, GAO/PEMD-10.1.4 (Washington, D.C.: March 1991). 
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Table 1 : GAO Assessment of FEMA's Implementation of Sound Management 
Practices for TFER 



Sound management practice 



GAO assessment 



1 . Develop objectives that link to the goals 
of the program and clearly articulate 
standards for determining pilot-program 
performance 



FEMA developed objectives for the states 
to follow that reflected TFER's goals but did 
not document the standards for determining 
whether TFER was successful. 



2. Develop processes and procedures for 
approving, reporting, and monitoring and 
provide program support 



FEMA specified support it would provide to 
the pilot program and procedures to 
monitor the pilot, but did not always follow 
these procedures. 



3. Develop and implement a data collection 
and data analysis plan 



FEMA developed elements of a data 
collection plan, but did not always ensure 
data collection tools would produce reliable 
results. Further, FEMA did not develop a 
data analysis plan to describe how it would 
track TFER's performance and evaluate its 
effectiveness. 



4. Ensure evaluation results are 
communicated to stakeholders 



FEMA has not yet decided whether, how, 
or when the TFER evaluation results will be 
shared with stakeholders. 



Source: GAO analysis of sound management practices and FEMA information. 



FEMA Developed 
Objectives that Reflect the 
Goals of the Pilot Program, 
but Did Not Document 
Standards for Determining 
Pilot Program Success 



FEMA developed objectives for the pilot states to follow, but did not 
document standards for determining whether TFER was successful. 
According to sound management practices, having objectives that reflect 
the goals of the pilot program can help determine whether the objectives 
of the pilot have been met. FEMA outlined three objectives for the TFER 
pilot program — develop plans, build relationships, and document lessons 
learned — and these objectives reflect TFER's goals. For example, the 
"develop plans" and "document lessons learned" objectives reflect the 
goals of TFER because achieving these objectives may help assess, 
strengthen, and advance state catastrophic preparedness planning — a 
stated TFER goal. Further, achieving the "build relationships" objective 
may enhance the stated TFER goal of integrating state planning efforts 
with those of federal agency partners. However, FEMA did not define how 
states' implementation of the objectives would impact the decision to 
extend or end TFER. According to sound program management practices, 
defining standards for determining pilot-program performance is 
necessary to determine what success level is appropriate forjudging the 
pilot program's effect. The TFER program manager said that discussions 
about the expectations for TFER were held at a high level with senior 
leadership from FEMA, DOD, and the participating states, such as the 
Adjutants General, before TFER planners were hired. However, FEMA did 
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not include the standards resulting from these conversations in the TFER 
program guidance. The program manager said that the agency had a 
limited time frame in which to design the pilot and that if he were to 
design the TFER pilot now, he would build language into the grant 
guidance that included standards for measuring performance. By clearly 
defining standards for determining program performance, FEMA would be 
better positioned to determine whether TFER enhanced states' 
catastrophic planning, as well as achieved a fuller integration of state 
planning efforts with federal agency partners through a deliberate 
blending of civil-military expertise. 



FEMA Outlined Steps to 
Administer TFER, but Did 
Not Always Adhere to Its 
Processes and Procedures 



FEMA outlined steps for administering TFER, which aligns with sound 
management practices, but FEMA did not always adhere to the processes 
and procedures it developed for the pilot. Sound management practices 
include developing processes and procedures for approving, reporting, 
and monitoring and providing program support. Further, according to 
Standards for Internal Control in the Federal Government, such control 
activities are an integral part of an entity's planning, implementing, 
reviewing, and accountability for stewardship of government resources 
and achieving effective results. FEMA outlined steps in two key 
documents, the TFER Guidance and Application Kit and the TFER Pilot 
Information Package in which FEMA described the support it would 
provide the participating states and the procedures for monitoring the 
program. 

FEMA developed processes and procedures for providing funding, 
training, holding a meeting with the pilot states, and approving state 
project management plans that met certain requirements, and establishing 
a mechanism for sharing lessons learned among the states, but FEMA did 
not fully follow the steps it outlined. For example, FEMA provided $1.75 
million in funds to the state administrative agencies identified in the TFER 
Pilot Information Package and established a virtual communication tool 
for the TFER pilot states to communicate and share lessons learned. 
However, FEMA provided training for the initial cohort of TFER planners 
in April 2009, which FEMA officials told us consisted of two-thirds of the 
planners originally hired by the states. All five states found FEMA's 
training useful. However, officials in two states told us additional training 
would have been useful. For example, one TFER planner who was not 
hired at the time of the training stated that additional training could have 
provided planners more opportunities to network and gain knowledge 
about state and federal resources. Further, FEMA conducted a meeting 
with the pilot states to accomplish tasks such as establishing support 
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requirements and planning deliverables, but FEMA approved three state 
project management plans that did not contain all the recommended 
information, such as specific tasks, milestones, and deliverables. The 
TFER program manager stated that part of the intent of TFER was to 
design it in real time, working with the jurisdictions, and that the 
omissions of the recommended information did not negatively impact the 
states' implementation of TFER. However, the states whose project 
management plans did not include specific deliverables have not 
completed draft plans. 

Further, FEMA also established processes and procedures for reporting 
and monitoring, but did not fully follow them. Specifically, FEMA stated in 
the TFER Pilot Information Package that the project manager would 
conduct biweekly conference calls with the states, but state officials in all 
five pilot states noted that FEMA headquarters involvement was 
inconsistent. According to the program manager, the purpose of these 
meetings was to ensure the pilot was functioning smoothly, so FEMA 
conducted biweekly meetings as TFER became operational but the 
meetings were discontinued once the pilots were underway. However, 
officials in four states reported that it would have been helpful if FEMA 
had provided more guidance on plan development. Further, TFER 
planners in one state reported that the lack of guidance made it more 
difficult to write plans. Another planner stated that FEMA did not provide 
the expected level of meetings and technical assistance. The program 
manager stated he was available to provide the states with assistance as 
needed throughout the pilot program. In addition, FEMA established 
reporting requirements and a plan to conduct a 6-month evaluation. For 
example, the states submitted reports to FEMA — to include the status of 
milestones and deliverables — but the program manager told us he did not 
review all of these reports because not all of these reports were submitted 
to his office. As a result, the program manager could not monitor progress 
in meeting milestones and completing deliverables. FEMA also conducted 
a 6-month evaluation of the pilot program, but four pilot states reported 
that FEMA did not provide feedback on the evaluation. According to 
sound management practices, continuous monitoring can be a useful tool 
for facilitating effective management by providing regular feedback about 
how well a program is performing its functions. This type of feedback 
allows managers to take corrective action when problems arise and can 
also provide stakeholders with regular assessments of program 
performance. By adhering to the steps outlined in TFER documents, 
FEMA could have more effectively monitored the pilot program to better 
ensure that the states receive the level of guidance and support they 
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identified as beneficial in order to enhance their TFER catastrophic 
planning efforts. 



FEMA Took Steps to 
Collect and Analyze Data 
from States to Evaluate 
TFER, but Did Not Always 
Include Elements of Sound 
Management Practices 



FEMA developed a data collection plan but did not reliably collect data for 
the pilot evaluation, and FEMA did not develop a data analysis plan to 
describe how it would track TFER's performance and evaluate its 
effectiveness based on standards to determine program performance. For 
example, FEMA developed elements of a data collection plan, following 
sound management practices such as detailing the type of data necessary 
to monitor and evaluate the pilot and identifying the timing and frequency 
of data collection. Further, FEMA's data collection plan followed reporting 
requirements specified in its program documents for TFER participants, 
and also included a data collection tool for evaluation of the pilot, 
presented in appendix III. However, FEMA did not always identify the 
source that was to collect the data. Sound management practices say that 
the source of data should be specified in the data collection plan, and 
according to program evaluation guidance, the reliability of evaluations 
can vary depending on who responds. During interviews with officials 
from participating states and FEMA regional offices, we found 
inconsistencies in who completed the evaluation questions. For example, 
two pilot states developed their own responses and three FEMA regions 
developed the responses for their respective pilot states, even though one 
of the questions explicitly asked for the opinion of state stakeholders. 
Further, two of the three states were not cognizant of the fact that their 
FEMA regional offices had completed the evaluation questions; therefore, 
these states' views may not have been accurately captured in the 
evaluation responses. Moreover, officials in one FEMA regional office 
questioned whether they were the appropriate FEMA personnel to 
administer the evaluation questions, since FEMA headquarters had 
collected all prior information from the states. Collecting data from 
consistent sources across the five pilot states could have better positioned 
FEMA to ensure the data were reliable and compare responses across 
states in order to draw conclusions about states' experiences in the pilot 
program. 



Further, FEMA did not collect information to determine whether all of the 
pilot program objectives were fully met. Specifically, FEMA changed some 
of the data collection questions it provided the states in the TFER Pilot 
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Information Package when it released updated questions in the TFER 
Evaluation Criteria document. 27 For example, FEMA used the same three 
objectives in both its original and updated documents, and its objectives 
on developing plans and documenting lessons learned objectives were 
consistent across documents, but FEMA did not collect information to 
assess all of the relationships it identified as important for its objective on 
building relationships in the TFER Pilot Information Package. For 
instance, FEMA did not evaluate coordination with FEMA, Defense 
Coordinating Officers, or Emergency Preparedness Liaison Officers — all 
relationships principal to implementing TFER, according to the TFER 
Pilot Information Package document. FEMA included a question that 
asked the pilot states which federal agencies they coordinated with, but 
did not include any question that would allow it to evaluate the extent to 
which coordination occurred with these stakeholders. As a result, the 
evaluation responses we reviewed listed the stakeholders they 
coordinated with but did not discuss the extent of coordination between 
state officials and stakeholders. 28 FEMA officials said that because the 
Defense Coordinating Officer and Emergency Preparedness Liaison 
Officer positions were not the only relationships explored between states 
and federal agencies, the evaluation criteria was changed to make it more 
general to determine with which agencies the TFER states engaged. 
However, by changing the evaluation question to a general and descriptive 
question on federal relationships, it will be difficult for FEMA to determine 
if the states achieved a fuller integration of planning efforts with federal 
agency partners — one of the goals of TFER. 

FEMA did not develop a data analysis plan to describe how it would track 
TFER's performance and evaluate its effectiveness based on criteria to 
determine program performance. According to sound management 
practices, a data analysis plan sets out who will do the analysis and when 
and how data will be analyzed to measure the pilot program's 
performance. FEMA did not finalize a data-analysis methodology 
describing how FEMA would use information collected from the states to 
evaluate TFER until after it collected responses to FEMA's evaluation 
criteria questions. The TFER program manager stated that FEMA plans to 



FEMA described the general objectives and proposed criteria in its December 2008 TFER 
Pilot Information Package document and provided more specifics in its June 2010 TFER 
Evaluation Criteria document. 

28 We reviewed evaluation responses from two of the pilot states. 
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look for trends in the states' responses to the evaluation questions to 
discern common threads and outliers. According to the program manager, 
FEMA did not develop a plan that outlined the overall approach it would 
use to analyze the TFER data during design of the pilot program because 
the pilot was not intended to produce quantitative results. He further 
stated that FEMA wanted to see what worked and what did not work as 
the pilot evolved. However, a data analysis plan could have helped guide 
the analysis of the qualitative data FEMA collected from the pilot states 
and FEMA regional offices. Developing data collection and data analysis 
plans following sound management practices to establish a methodology 
for collecting and evaluating the pilot's results could have provided FEMA 
with reasonable assurance that it possessed the data and performance 
information needed to draw reasonable conclusions on the impact of 
TFER. 

The TFER program manager told us he is analyzing the data collected for 
TFER to draw conclusions regarding the pilot program, but FEMA officials 
stated that TFER would not be continued past the pilot stage. The 
information on the TFER pilot program's evaluation stated that the results 
of the evaluation would determine the prospect for the program's 
continuation and level of future funding. However, prior to conducting its 
evaluation of the pilot, FEMA announced in March 2010 that TFER would 
not be continued as a stand-alone grant program. According to FEMA 
officials, the decision to discontinue TFER as a stand-alone grant program 
was made because DOD may begin a similar initiative and states could 
allocate other FEMA grant funding for catastrophic planning purposes. 
Specifically, the TFER program manager noted that the National Guard 
Homeland Response Force, which is a DOD concept that would place 
National Guard personnel in each FEMA region, could continue the 
planning efforts associated with TFER. 29 DOD officials stated that these 
personnel would be located regionally and not in individual states; 
therefore, the National Guard Homeland Response Force initiative could 
supplement but not replace a TFER-like initiative. Also, according to DOD, 
the National Guard Homeland Response Force would have as its mission 



According to a September 2010 report, "the purpose of these personnel is to provide a 
regional response capability; focus on planning, training and exercising; and forge strong 
links between the federal level and state and local authorities." Before Disaster Strikes: 
Imperatives for Enhancing Defense Support to Civil Authorities. Advisory Panel to 
Department of Defense Capabilities for Support of Civil Authorities After Certain Incidents 
(Sept. 15, 2010). 
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response to chemical, biological, radiological, nuclear, and high-explosive 
incidents and would not have a broad purview like TFER. A FEMA 
assistant administrator also said that FEMA made the decision to not 
continue TFER due to changing budget conditions and to focus on its 
legislative and presidential directive mandates. Other FEMA grant 
programs available for catastrophic planning are discussed in appendix II. 



FEMA Has Not Committed 
to Sharing the TFER 
Evaluation Results with 
Stakeholders 



According to the TFER program manager, FEMA leadership will make a 
decision on whether and how the results of the evaluation are 
disseminated after the program office finishes the evaluation of TFER. 30 
According to sound management practices, results and recommendations 
that emerge from evaluations must be disseminated in ways that meet the 
needs of stakeholders. The TFER program manager stated it is not his 
decision whether the results of the pilot's evaluation will be shared. 
According to the TFER program manager, FEMA leadership will make a 
decision on whether and how the results of the evaluation are 
disseminated after the program office finishes the evaluation of TFER. 
FEMA administered the TFER pilot program, but other key stakeholders in 
federal disaster response, such as the pilot states and DOD, could benefit 
from the results of FEMA's evaluation of the TFER pilot program. 
Disseminating the results, recommendations, and lessons learned from the 
TFER pilot could assist future state catastrophic planning efforts, whether 
they be led by FEMA, DOD, another federal agency, or a state. 



FEMA Does Not Have 
Clear Policy Guidance on 
Designing, Administering, 
and Evaluating Pilot 
Programs 



FEMA's Office of Policy and Program Analysis reported that FEMA does 
not have a directive on how to design, administer, and evaluate a pilot 
program. In the absence of this, FEMA does not have a systematic 
approach for designing, administering, and evaluating such programs. The 
program manager stated there are no formal guidelines for developing 
FEMA grants. We previously reported that FEMA faced methodological 
challenges in assessing capabilities and has not generated meaningful 



In January 2011, FEMA said the evaluation would be completed in February 2011, but as 
of March 2011 FEMA had not completed its evaluation of TFER. 
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preparedness information from data collected from pilot programs. 31 
Standards for Internal Controls in the Federal Government state that 
management is responsible for ensuring that detailed policies, procedures, 
and practices are developed and built into and are an integral part of 
operations. 32 These elements, which are an integral part of an agency's 
ability to ensure accountability and achieve effective results, need to be 
clearly documented to help ensure that management directives are carried 
out as intended. By developing and implementing policy guidance that 
includes sound management practices, FEMA could be better positioned 
to ensure its pilot programs meet their intended goals. 



All Pilot States Have 
Made Progress in 
Addressing the Pilot 
Program Objectives, 
but No State Has 
Fully Addressed All 
Three Objectives 



All five states have taken steps to follow FEMA's TFER Information 
Package and TFER Evaluation Criteria document to address the TFER 
pilot program's three objectives — develop plans, build relationships, and 
document lessons learned — and these efforts are ongoing. 33 First, of the 
four states committed to developing plans, two states have completed 
draft plans, and TFER planners in all states reported using FEMA's CPG 
101 planning process to develop initiatives that fill gaps in state 
catastrophic planning. Second, all five states built relationships with 
stakeholders, including FEMA and state agencies, but coordination with 
DOD — specifically, with the Defense Coordinating Officers and 
Emergency Preparedness Liaison Officers — was limited. Third, all states 
documented lessons learned from the pilot program, which included the 
benefit of dedicated funding for catastrophic planning, and establishing 
longer time frames for states to complete the planning process. However, 
these efforts are ongoing as four states continue to spend TFER funds. 



See, GAO, National Preparedness: FEMA Has Made Progress, but Needs to Complete 
and Integrate Planning, Exercise, and Assessment Efforts, GAO-09-369 (Washington, 
D.C.: Apr. 30, 2009). For example, the National Preparedness System was piloted in 10 
states and discontinued because it was time consuming and did not produce meaningful 
data. In response to our recommendations, FEMA said it has established a working group 
to help consolidate and streamline reporting requirements for state, tribal, and local 
stakeholders. 

32 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 
(November 1999). 

33 FEMA announced it is not continuing TFER past the pilot stage, but states continue to 
draw down remaining funds and complete activities initiated under TFER. 
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Four of the states are developing plans and all five pilot states reported 
they are using the planning process outlined in FEMA's CPG 101. 
However, two of the four states developing plans have not completed 
them. The fifth state is focusing on other planning initiatives. FEMA 
articulated the criteria for this objective in the TFER Pilot Information 
Package and the TFER Evaluation Criteria document, which focused on 
organizing plans according to CPG 101 and plan development. Based on 
audit work conducted in the TFER pilot states, we assessed the states' 
progress based on these criteria, and concluded that two of the states have 
made substantial progress in fulfilling the FEMA objective on developing 
plans, and the other three have made some progress, as shown in figure 4. 
Figure 4 further explains this assessment, as well as appendix IV, which 
provides specific information regarding each pilot state's TFER initiatives 
and progress made towards meeting FEMA's objectives for TFER. 

• Description of efforts to integrate plans with 
state and federal partners 

• Planning that reflects the National 
Preparedness Guidelines 

• Planning responsive to the National 
Planning Scenarios and other threats based 
on the state's all-hazards risk assessment 

• Planning consistent and integrated with the 
state's emergency plans 

• Development of actual plans 

Source: GAO application of FEMA criteria. 



Two of Five States Have 
Made Substantial Progress 
in Developing Plans, and 
TFER Planners in All 
States Reported Using 
FEMA's Planning Process 
to Develop Initiatives that 
Fill Gaps in State 
Catastrophic Planning 

TFER Objective 1 : Develop Plans 

Assessing a state's progress in this objective 
was based on efforts made to address the 
following components, among others: 
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Figure 4: Assessment of Pilot States' Plan Development Through an Effective Planning Process, as of March 2011 " 



State 


GAO 
assessment 


Status 


Hawaii 


r\ 

w 


State officials reportsd thatTFER planning integrates with the state's existing hurricane planning efforts, which relates to a 
National Planning Scenario. State officials told us that TFER planners are following the steps outlined in CPG 1 01 , but have not 
completed the full cycle of planning steps for its plans, as Hawaii has not completed draft plans. 


Massachusetts 


Q 


Massachusetts focused on initiatives pertaining to a hurricane and other priorities identified by state emergency management 
officials and TFER planners. However, state officials told us only one of the initiatives specifically pertained to a National Planning 
Scenario. State officials said the planners followed the steps outlined in CPG 1 01 , but could not complete the full cycle of 
planning because state officials decided that the Massachusetts TFER planners would coordinate the planning efforts but plans 
would be completed by other state agencies. 


South Carolina 


• 


South Carolina focused on the National Planning Scenarios related to terrorist attacks. The state has completed a revision of its 
Catastrophic Incident Annex and it has completed drafts of its Terrorism Prevention and Response Plan and three other plans 
related to terrorist attacks, which officials told us fill gaps in state planning. Planners reported they are following the steps 
outlined in CPG 1 01 , but have not finished the full cycle of planning for any of its plans, as South Carolina has not exercised 
them. 


Washington 


• 


Washington selected as its primary National Planning Scenario a catastrophic earthquake and accompanying tsunami. 
Washington integrated TFER logistics plans and a catastrophic incident plan with existing state plans and filled gaps in the state 
plan compendium, according to state officials. The state has finalized two of its seven plans and has final drafts for three others. 
Washington officials told us they are following CPG 101 planning process and are structuring plan content accordingly. 


West Virginia 


Q 


West Virginia identified three possible scenarios: a chemical incident, a mass evacuation of the National Capital Region, and a 
dam failure. These events could be caused by a hurricane or terrorist attack, which pertain to several National Planning 
Scenarios. According to state officials, West Virginia's TFER planning efforts fill gaps in catastrophic planning and include 
development of a catastrophic incident plan. State officials said that planners are following the steps outlined in CPG 101 , but 
have not finished the full cycle of planning for any of its products, as West Virginia has not completed draft plans. 



• Substantial progress 
© Some progress 
O Little to no progress 



Source: GAO analysis of state information. 

"The GAO assessment was assigned based on an average of five components, as discussed in 
appendix I and elaborated in appendix IV. 

Developing plans. The status of the states' TFER plans varies; Washington 
and South Carolina have completed some but not all of their plans, while 
Hawaii and West Virginia have not completed any draft plans. Hawaii and 
West Virginia officials identified reasons for not having completed draft 
plans, including delays in hiring qualified planners. One of FEMA's 
requirements for TFER was that planners have experience in conducting 
civil-military support planning and operations with the National Guard 
and/or military experience. In Massachusetts, South Carolina, and 
Washington, all planners initially hired for TFER had military 
backgrounds. These states hired planners as early as February 2009 — 5 
months after the pilot program began. One of the planners initially hired 
had prior military and civilian experience, and the other had a logistics 
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background. 34 Hawaii hired its first TFER planner in April 2009 but did not 
hire a second planner until August 2009 — 11 months after the pilot began. 
Hawaii officials told us that they found it difficult to hire qualified 
individuals with a military background because better paying options exist 
in the state for qualified individuals. One of Hawaii's two current planners 
does not have military experience, but state officials told us that the 
individual is qualified because of his information technology background, 
which was needed to support logistics planning. Additionally, he worked 
for the state planning agency. West Virginia officials reported that they 
hired their planners in the July/August 2009 time frame — 10 to 11 months 
after the pilot began. Two applicants originally applied for the position, 
one of which West Virginia hired. This individual did not have a military 
background, but officials stated he was qualified based on his decades of 
experience as a first responder. West Virginia subsequently hired a 
National Guardsman. The West Virginia TFER supervisor stated that 
inadequate planning capacity at the local level also delayed TFER efforts 
because West Virginia TFER focused on building capacity at the local 
level. FEMA granted all five states a 6-month extension based on delays 
the states experienced. As previously discussed, planning continues in 
four states, as they are still spending TFER funds, as of March 2011 (see 
fig. 5). 



This second planner worked from August 2009 to February 2010. 
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Figure 5: TFER Funds Spent by State, as of March 2011 
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Source: GAO analysis of state-provided data. 



Massachusetts TFER planners facilitated planning between different levels 
of government, but other state planners are developing the plans. 
Massachusetts initially focused on contributing to Boston's improvised 
explosive device planning efforts, but differences in time lines, combined 
with TFER's brief performance period, caused the team to change its focus 
to facilitating collaborative emergency planning among federal, state, 
local, and private stakeholders. For example, the Massachusetts TFER 
planners facilitated collaborative planning among the Massachusetts 
Department of Public Health, the U.S. Department of Health and Human 
Services, the Massachusetts Emergency Management Agency, and FEMA 
to create a statewide medical support plan. TFER planners developed 
interagency relationships and helped the planning team select goals and 
objectives but did not draft the plan, which state officials said other state 
planners are developing. State officials decided the TFER planners would 
not actually write the plans because of the pilot program's short time 
frame. 
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Filling gaps in state planning. State officials reported that TFER 
initiatives fill gaps in existing state plans. For example, South Carolina, 
Washington, and West Virginia chose to work on a catastrophic incident 
plan as part of their TFER efforts, and this plan is an annex to the states' 
basic emergency operations plan. Further, officials in four of five states 
reported that TFER initiatives focused on gaps not addressed by other 
planning initiatives. For example, the FEMA Region X TFER contact 
stated that Washington successfully integrated TFER with the Regional 
Catastrophic Grant Planning Program. Two of three West Virginia TFER 
scenarios pertain in part to scenarios addressed by other planning efforts, 
but officials stated that TFER planners work collaboratively with the 
planners involved in these other efforts, as described in appendix IV. 
Additionally, Hawaii officials anticipate the Regional Catastrophic 
Planning Grant Program will address hurricanes in 2011 — before the state 
exhausts its TFER funds. State officials expect these efforts to be 
complementary. 

FEMA's planning process. All five states reported that they are following 
FEMA's planning process, and states that developed draft plans utilized 
the structure recommended in FEMA's CPG 101 (see fig. 6). All of the 
states reported they are following FEMA's six-step planning process, but 
they are in different stages of that process, as of March 2011. 



Figure 6: Furthest Step Each Pilot State Achieved in the CPG 101 Planning Process, as of March 2011 
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Source: GAO analysis of Comprehensive Preparedness Guide 101 and interviews with state officials. 



Washington has exercised three out of seven of its plans — part of step six. 
However, most of Washington's plans remain in draft form, which is 
associated with step five. Since Hawaii, South Carolina, and West Virginia 
officials told us their states have not exercised any plans, they are in step 
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five — the step where planners write the plan and the plan is approved. 
Massachusetts state officials decided that their pilot program would 
coordinate planning efforts, but that other state planners would develop 
plans; officials stated they did not move beyond step four in the planning 
process. 

Officials in the four states that committed to developing plans reported 
that they are structuring plans as recommended by CPG 101, but some 
sections are not included in the states' draft plans. CPG 101 provides a 
recommended structure for annexes to a state's basic emergency 
operations plan. 35 These annexes comprise half of the 32 TFER initiatives 
spanning all five pilot states. Ten of 13 annexes we reviewed were missing 
at least one recommended section. 36 For example, none of the three 
catastrophic incident plans we reviewed contain a section on direction, 
control, and coordination. South Carolina and Washington officials noted 
that their plans did not include all of the recommended sections because 
other state plans like the basic emergency operations plan contained these 
sections. Washington officials further stated that TFER-supported 
planning is part of the state emergency operations plan and is not designed 
to stand alone. Further, Hawaii officials told us that they are following the 
FEMA guidance, but its plans are incomplete and work is ongoing. West 
Virginia officials stated that it is following the structure recommended in 
CPG 101, but since some of the sections rely on information gathered at 
the county level, it will take about 5 years before the state has enough 
information to include each recommended section in its catastrophic 
incident plan. We were unable to further assess the content of the pilot 
states' plans because all but three of the plans TFER states are developing 
remain in draft form. 



3 CPG 101 recommends including the following sections in supporting annexes to the state 
basic emergency operations plan: purpose, situation overview, and planning assumptions; 
concept of operations; organization and assignment of responsibilities; direction, control, 
and coordination; information collection and dissemination; administration, finance, and 
logistics; and authorities and references. 

36 We were unable to review the other annexes because states were not at the point in the 
planning process to provide us with draft plans. 
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All Five States Made 
Progress in Building 
Relationships with 
Stakeholders such as 
FEMA and State Agencies, 
but Coordination with 
Federal Military 
Stakeholders Was Limited 

TFER Objective 2: Build Relationships 

Assessing a state's progress in this objective 
was based on efforts made to address the 
following components: 

• Planning that reflects the formation of 
integrated planning teams described in the 
Comprehensive Preparedness Guide 101 

• Establishment or enhancement of mecha- 
nisms to ensure multijurisdictional and 
interagency planning coordination 

• Plans coordinated with local, state, and 
federal agencies 

• Full use of civil-military planning expertise to 
assess, strengthen, and advance state 
catastrophic planning 

• Incorporation of private sector capabilities 

• Coordination with other states 

• Coordination with FEMA 

Source: GAO application of FEMA criteria. 



All five states participating in the TFER pilot program reported that they 
built relationships with stakeholders through collaborative planning teams 
and coordinated with various stakeholders. State officials told us they 
coordinated with FEMA, other state agencies, and the National Guard, but 
none of the states coordinated regularly with federal military stakeholders 
such as Defense Coordinating Officers and Emergency Preparedness 
Liaison Officers as FEMA recommended in the TFER Pilot Information 
Package. FEMA articulated the criteria for this objective in the TFER Pilot 
Information Package and the TFER Evaluation Criteria document, which 
focused on forming planning teams and building relationships with 
relevant stakeholders. As discussed in figure 7, all five states have made 
some progress in building relationships with relevant stakeholders. Our 
assessment — based on audit work conducted in the TFER pilot states — is 
summarized in figure 7 and further explained below as well as in 
appendix IV, which provides specific information regarding each pilot 
state's TFER initiatives and progress made towards this objective. 



Figure 7: Assessment of Pilot States' Progress in Building Relationships with 
Stakeholders, as of March 201 Y 



State 


GAO 
assessment 


Status 


Hawaii 


w 


Hawaii developed planning teams that included state agencies and 
relevant stakeholders, but state officials told us they had little contact 
with DOD entities. 


Massachusetts 


w 


Massachusetts TFER planners helped formulate planning teams that 
worked to improve the state's catastrophic preparedness, and included 
planners from state agencies, as well as relevant stakeholders. State 
officials told us that contact with DOD entities was minimal. 


South Carolina 


Q 


South Carolina inserted its TFER planners into existing planning 
teams in its law enforcement and emergency management divisions 
that also included other state agencies and relevant stakeholders. 
According to state officials, TFER planners coordinated with the DOD 
entities, but this coordination was limited. 


Washington 


Q 


Washington officials stated they developed planning teams that 
included planners from state agencies, and relevant stakeholders. 
According to state officials, DOD entities were minimally involved. 


West Virginia 


Q 


West Virginia officials said they developed planning teams that 
included state agencies, the National Guard, and private sector 
stakeholders. State officials told us coordination with DOD entities was 
minimal. 



9 Substantial progress 

© Some progress 

O Little to no progress 

Source: GAO analysis of state information. 

"The GAO assessment was assigned based on an average of seven components, as discussed in 
appendix I and elaborated in appendix IV. 
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Four states reported that their TFER planning teams worked efficiently. 
One state, West Virginia, identified problems with its planning team. West 
Virginia officials told us they faced challenges coordinating and integrating 
plans with some stakeholders. For example, some state and local 
stakeholders resisted coordination and overestimated their planning 
capacities. All five pilot states included federal, state, local, and private 
stakeholders in the planning process, and utilized work groups to enhance 
these relationships. 

All five states reported that they coordinated with FEMA and other states, 
but the level of coordination was inconsistent throughout the pilot. All 
states reported they benefited from coordination with FEMA headquarters 
in the beginning of the pilot through training and/or teleconferences. 
Further, four states identified their relationship to the FEMA region as 
beneficial during the pilot program. A TFER planner in the fifth state 
received some needed information from its region but the state's TFER 
program supervisor told us they coordinated infrequently and that regional 
support was inadequate. However, as previously discussed, all states 
reported that coordination with FEMA headquarters waned throughout 
the pilot. Further, Washington TFER planners coordinated with FEMA, but 
officials told us that coordination could be improved. Specifically, state 
officials reported that FEMA Region X participated in work groups and 
provided suggestions on planning, but FEMA headquarters provided 
limited guidance on the implementation of the pilot program, and grant 
guidance was inconsistent. According to FEMA's TFER program manager, 
FEMA headquarters was more involved at the beginning phase of the 
TFER to help set up the pilot program, but states needed less assistance 
after that point. 

In addition to FEMA's guidance that TFER planners have experience in 
conducting civil-military support planning and operations with the 
National Guard and/or military experience, FEMA also encouraged TFER 
states to coordinate with National Guard and/or federal military entities 
throughout the pilot program. We found that all TFER states coordinated 
with the National Guard to varying degrees, but coordination with federal 
military entities was limited in all pilot states. In Massachusetts and West 
Virginia, state officials placed at least one TFER planner under National 
Guard management, which facilitated collaboration, according to state 
officials. In Hawaii and Washington, the state Adjutant General oversees 
both the National Guard and the state emergency management agency, 
which facilitated coordination. However, Hawaii and Washington officials 
stated that coordination with the National Guard was limited because of 
deployments. A TFER planner in South Carolina told us the state agencies 
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involved in TFER coordinated with National Guard officials on a monthly 
basis and included them in TFER workgroups. However, a South Carolina 
National Guard official told us the National Guard could have provided 
more input into the TFER products South Carolina produced. 

FEMA's TFER Pilot Information Package recommended the pilot states 
include military entities as full partners in the planning process. At the 
onset of TFER, DOD provided training to planners and state officials on 
civil-military planning integration and how NORTHCOM could support 
TFER efforts. State officials in two states told us DOD officials were 
involved in TFER work groups and officials in another state reported that 
DOD officials provided them with contacts and information needed to 
develop their plans, but none of the states coordinated regularly with the 
federal military Defense Coordinating Officers or Emergency 
Preparedness Liaison Officers. In no instance did TFER pilot states ask 
Emergency Preparedness Liaison Officers or Defense Coordinating 
Officers to help develop or comment on plans, according to state and DOD 
officials. DOD stakeholders are important in the planning process because 
DOD may be asked to provide response capabilities in the event of a 
catastrophic incident. State officials told us they needed to draft the plans 
first, before soliciting DOD's input. One of the Defense Coordinating 
Officers we spoke with agreed that there was little he could do to assist 
TFER states until the planners drafted plans. Further, state officials told us 
the availability of the Emergency Preparedness Liaison Officers — part- 
time military personnel who work primarily on weekends — limited 
coordination with state officials and TFER planners. State officials and a 
Defense Coordinating Officer told us this work schedule is not conducive 
to frequent coordination between Emergency Preparedness Liaison 
Officers and state officials, who work during the week. Further, DOD 
officials in two regions stated that DOD entities could have helped TFER 
planners integrate state and federal plans. 
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All Five Pilot States Have 
Made Some Progress in 
Documenting Lessons 
Learned, but Four States 
Have Not Exercised Plans 
to Evaluate TFER's 
Effectiveness 



TFER Objective 3: Document Lessons 
Learned 

Assessing a state's progress in this objective 
was based on efforts made to address the 
following components, among others: 

• Development of after-action reports from 
exercises or assessment documents during 
TFER to evaluate the effectiveness of the 
pilot 

• Provide recommendations for improvement 
to increase the effectiveness of catastrophic 
planning programs 

Source: GAO application of FEMA criteria. 



All states have documented lessons learned to date, but four states have 
not exercised TFER plans to determine their effectiveness in the event of 
an emergency. As shown in figure 8 and further explained in appendix IV, 
the TFER pilot states have provided a number of lessons learned and 
recommendations on how to improve a similar program in the future. 37 
One state, Washington, has made substantial progress documenting 
lessons learned because it has exercised three of its plans to evaluate their 
effectiveness, one of the components FEMA cited in the objective. FEMA 
articulated the criteria for this objective in the TFER Pilot Information 
Package and the TFER Evaluation Criteria document, which focused on 
documenting lessons learned, including exercising plans to evaluate 
TFER's effectiveness. All five pilot states documented lessons learned for 
a report published by the National Guard Bureau in August 2010, and we 
obtained additional lessons learned during our interviews with state 
officials. 38 Further, two of the states provided us with the lessons learned 
they submitted to FEMA for its evaluation, which is not yet complete. 39 



The states provided lessons learned to the National Guard Bureau Strategic Advisory 
Group Leadership Council, which are described in the August 2010 report to the group; to 
GAO in interviews with state officials; and to FEMA in evaluation forms. 

38 The National Guard Bureau, Task Force for Emergency Readiness (TFER) Program: 
Report to the National Guard Bureau Strategic Advisory Group Leadership Council 
(Washington, D.C., August 2010). 

39 FEMA regional offices responded for the remaining three states. 
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Figure 8: Assessment of Pilot States' Documentation of Lessons Learned, as of March 2011 " 



State 


GAO 
assessment 


Status 


Hawaii 


Q 


Hawaii officials provided recommendations, including: continue and expand the program to other states, maintain flexibility to 
determine work plans, and dedicate funding for catastrophic planning. Additionally, state officials recommended that future 
catastrophic planning programs have longer time frames than the TFER pilot program, provide more clearly defined guidance, 
and greater coordination between FEMA and DOD. Further, funding was not enough to provide competitive salaries. Hawaii has 
not exercised any of its plans to determine if they will be effective in the event of an emergency. 


Massachusetts 


Q 


Massachusetts officials recommended that FEMA continue and expand the program to other states, maintain flexibility to 
determine work plans, and dedicate funding for catastrophic planning. Officials stated the TFER planners all had some level of 
military experience, which helped them cope with the logistical challenges, and have credibility with certain audiences. 
Additionally, officials recommended that future catastrophic planning programs have longer time frames than the TFER pilot 
program and provide more clearly defined guidance. According to state officials, they have not exercised any plans. 


South Carolina 


Q 


South Carolina provided a number of recommendations, including: continue and expand the program, maintain flexibility to 
determine work plans, and dedicate funding for catastrophic planning. Additionally, South Carolina officials recommended that 
future catastrophic planning programs have longer time frames than the TFER pilot program, and that FEMA should provide 
more clearly defined guidance. Officials identified coordinating security clearances across DHS and DOD as a challenge that 
should be resolved in similar programs in the future. The sharing of lessons learned and techniques/procedures with other states 
would have been helpful in developing catastrophic plans through the TFER pilot program. South Carolina also cited good initial 
training opportunities through TFER; however these opportunities did not extend throughout the life of the program, and thus 
were not available to all South Carolina TFER staff. Officials stated they have scheduled an exercise for March 201 1 and that 
after-action reports will subsequently be developed. 


Washington 


• 


Washington provided recommendations, including: continue and expand the program to other states, maintain the broad scope 
of the projects eligible to be addressed by the TFER resources, and dedicate funding for catastrophic planning. Additionally, 
state officials recommended that future catastrophic planning programs have longer time frames than the TFER pilot program 
and FEMA provide more guidance. Further, FEMA could improve in management of grants like TFER. Washington has 
exercised three out of seven of its TFER plans. 


West Virginia 


Q 


West Virginia recommended FEMA continue and expand the program, maintain flexibility to determine workplans, and dedicate 
funding to hire planners. Additionally, officials recommended that the new program have longer time frames than the pilot and 
provide more clearly defined guidance. West Virginia reported there was significant difficulty in hiring people with the requisite 
military/civilian catastrophic planning skills and experience because of the short duration of the program. West Virginia has not 
exercised any of its plans. 



• Substantial progress 
© Some progress 
O Little to no progress 



Source: GAO analysis of state information. 

"The GAO assessment was assigned based on an average of two components, as discussed in 
appendix I and elaborated in appendix IV. 

Some of the lessons learned were similar across the pilot states. For 
example, all five states considered the pilot a success; Hawaii, 
Massachusetts, South Carolina, and West Virginia officials reported that 
TFER allowed their states to conduct catastrophic planning that otherwise 
would not have occurred, and Washington reported that TFER advanced 
its catastrophic and emergency logistics planning by at least 2 years. 
Washington officials further stated that the $350,000 the state received in 
TFER funding generated more return on investment than any other 
$350,000 in the state's emergency planning budget. Based on their view of 
TFER's success, officials in each of the five pilot states recommended 
FEMA expand TFER to other states. 
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All states reported three principal reasons for considering TFER a success: 
(1) autonomy to develop their own work plans, (2) provision of dedicated 
funds to hire planners, and (3) the background of the TFER planners. 

(1) State officials reported that they worked collaboratively with 
FEMA to select catastrophic scenarios. For example, South 
Carolina chose to work on plans related to terrorism attacks 
because state officials believe they had adequately planned for 
hurricanes and earthquakes — the two National Planning Scenarios 
ranked higher than terrorism attacks in the state's risk assessment. 
Officials in all five states highlighted the benefit of this flexibility. 

(2) Officials in all five states reported that the narrow focus of 
TFER, dedicated solely to enhance catastrophic planning, was key 
to the pilot program's success. State officials told us that state 
planners are often required to assist emergency planning and 
response efforts during more routine disasters, such as seasonal 
floods and forest fires, but since the terms of TFER did not allow 
states to use TFER funds for these efforts, catastrophic planning 
continued unabated. 

(3) Officials in all five states reported that the planners' military 
background was beneficial. For example, state officials in 
Massachusetts told us the planners' military experience provided 
them with a level of experience and training not often found in 
other sectors and gave them credibility. However, officials in three 
states told us that civilian planning or emergency management 
experience also provided TFER planners the needed skill set. 

States also reported challenges they faced, including (1) short time frames 
that limited their ability to complete the planning process, and (2) not 
enough guidance for the specialized plans they were developing. 

(1) All five states reported that the short time frames limited their 
ability to complete the planning process. FEMA structured TFER 
as an 18-month program, and states noted that this was an 
insufficient amount of time to complete the full cycle of planning. 
Officials in one state thought that an adequate time frame would 
have been 3 years, noting that this is the time frame FEMA 
established for the Regional Catastrophic Planning Grant 
Program. The TFER program manager stated that FEMA's 6- 
month program extension — bringing the pilot program time frame 
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to 2 years — provided a sufficient amount of time to complete the 
TFER initiatives. 

(2) CPG 101 provides general format and content guidance 
applicable to all plans, but officials in four of the pilot states said 
that it would have been helpful for FEMA to provide guidance for 
specialized plans such as catastrophic incident annexes, terrorism 
plans, and logistics plans. For example, one TFER planner stated 
that clearer guidance could have accelerated the planning 
process. Further, officials in a fourth state said that they are using 
an unpublished FEMA guide to support their planning efforts. 
FEMA officials told us that states can request technical assistance 
and examples of well-developed plans to aid in their own plan 
development. According to the TFER program manager, FEMA is 
in the process of developing additional Comprehensive 
Preparedness Guides that will inform plan content for future state 
planning efforts. 

The process of documenting lessons learned is ongoing because Hawaii, 
South Carolina, Washington, and West Virginia are still developing TFER 
plans and spending TFER funds. 



Hurricane Katrina highlighted gaps in the nation's preparedness to 
respond effectively to catastrophic incidents. By their nature, catastrophic 
events involve extraordinary levels of casualties, damage, or disruption 
that will likely immediately overwhelm state and local responders — 
circumstances that make sound planning for catastrophic events all the 
more crucial. Planning is a key component of national preparedness. 
Planning provides a methodology to determine required capabilities and 
helps stakeholders learn and practice their roles for building and 
sustaining national preparedness capabilities against terrorist attacks and 
other hazards. As state and local governments continue to develop and 
improve plans for catastrophic events and identify potential resource 
shortfalls, the federal government will be in a better position to 
understand the nature of the gaps it may be called upon to fill if state and 
local resources are overwhelmed. The TFER pilot program was one such 
effort to enhance catastrophic preparedness and provide federal 
stakeholders with valuable information regarding local, state, or regional 
response capability. However, by not consistently following sound 
management practices to design, administer, and evaluate the TFER pilot 
program and by not following its processes and procedures for 
administering the pilot, it is unclear whether TFER achieved its intended 



Conclusions 
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purpose. As we have reported, FEMA has piloted other efforts that have 
not generated meaningful preparedness information from the data 
collected. Future pilot programs at FEMA could benefit from policy 
guidance that includes sound management practices to design, administer, 
and evaluate pilot programs. In addition, future state catastrophic planning 
efforts could benefit from the dissemination of the evaluation results of 
the TFER pilot program to relevant stakeholders. 



Recommendations for 
Executive Action 



We recommend the Administrator, Federal Emergency Management 
Agency, take the following two actions. 

To help ensure future pilot programs achieve their intended results and 
provide the performance information needed to make effective 
management decisions for broader implementation, develop and 
implement policies and guidance for pilot programs that follow sound 
management practices. This guidance should include, at a minimum, 
requirements for 

• a clearly articulated methodology with objectives reflective of 
overall program goals and standards for determining program 
performance; 

• procedures for monitoring program performance; 

• a data collection plan; 

• a data analysis plan; and 

• a process to disseminate the results and lessons learned that 
emerge from the pilot. 

To help ensure stakeholders receive valuable information regarding 
catastrophic preparedness from lessons learned during the TFER pilot 
program, disseminate the evaluation results and recommendations that 
emerge from the TFER pilot program in ways that meet the needs of 
current and future stakeholders. 



Agency Comments 
and Our Evaluation 



We provided a draft of this report to FEMA and DOD for review and 
comment. FEMA provided oral comments on the draft report on April 5, 
2011. FEMA fully concurred with both of our recommendations but did 
not specify how it planned to address them. DOD did not provide 
comments on the draft report because the report did not include 
recommendations to DOD. Both FEMA and DOD provided technical 
comments, which we incorporated throughout our report as appropriate. 
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As agreed, unless you publicly announce the contents of this report earlier, 
we plan no further distribution until 30 days from the report date. At that 
time, we are sending copies of this report to the Administrator, Federal 
Emergency Management Agency, and Secretary of Defense. In addition, 
the report will be available at no charge on GAO's Web site at 
http://www.gao.gov. If you or your staff have any questions concerning 
this report, please contact Bill Jenkins at (202) 512-8777 or by e-mail at 
jenkinswo@gao.gov, or Davi M. D'Agostino at (202) 512-5431 or by e-mail 
at dagostinod@gao.gov. Contact points for our Offices of Congressional 
Relations and Public Affairs may be found on the last page of this report. 




Davi M. D'Agostino, Director 
Defense Capabilities and Management 




William O. Jenkins, Director 
Homeland Security and Justice 
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Appendix I: Scope and Methodology 



To identify the extent to which the Federal Emergency Management 
Agency (FEMA) followed sound management practices in developing, 
administering, and evaluating the Task Force for Emergency Readiness 
(TFER) pilot program, we analyzed program guidance and other key 
documents including the TFER Pilot Information Package and the 
Guidance and Application Kit. We analyzed prior GAO work on pilot 
programs and program management 1 as well as social science literature 2 
to determine elements of sound management practices. We considered 
other criteria, consulting checklists on elements of program management, 
but determined some of the characteristics identified in this literature 
were not appropriate or suitable to the type of study performed by FEMA. 
We grouped the elements of sound management practices that we 
identified as relevant into three categories: design, administration, and 
evaluation. We then analyzed the information obtained from our 
interviews and site visits and compared the results of our analysis with the 
identified elements of sound management practices. Furthermore, we 
analyzed prior GAO reports and Department of Homeland Security (DHS) 
Office of Inspector General reports on FEMA grants administration and 
catastrophic preparedness to identify any past challenges. 

We reviewed and analyzed FEMA grant program guidance and met with 
FEMA officials in the Grants Program Directorate to determine whether 
other grant programs, in addition to TFER, allow for funds to be used for 
catastrophic planning. We identified four major grant programs states can 
use for catastrophic planning purposes: the Regional Catastrophic 
Planning Grant Program, the State Homeland Security Program, the Urban 
Areas Security Initiative, and the Emergency Management Performance 
Grants. Other grants, such as the Port Security Grant Program (PSGP) 



1 See, GAO, Tax Administration: IRS Needs to Strengthen Its Approach for Evaluating 
the SRFMI Data-Sharing Pilot Program, GAO-09-45 (Washington, D.C.: Nov. 7, 2008). 
Aviation Security: A National Strategy and Other Actions Would Strengthen TSA's 
Efforts to Secure Commercial Airport Perimeters and Access Controls, GAO-09-399 
(Washington, D.C.: Sept. 30, 2009). Transportation Worker Identification Credential: 
Progress Made in Enrolling Workers and Activating Credentials but Evaluation Plan 
Needed to Help Inform the Implementation of Card Readers, GAO-10-43 (Washington, 
D.C.: Nov. 18, 2009). See also, GAO, Designing Evaluations, GAO/PEMD-10.1.4 
(Washington, D.C.: March 1991). 

2 P.H. Rossi, M.W. Lipsey, and H.E. Freeman, Evaluation: A Systematic Approach 
(Thousand Oaks, Calif.: 2004); Project Management Institute, The Standard for Program 
Management® (Newton Square, Pa: 2006); and Daniel L. Stufflebeam, Program Evaluations 
Metaevaluation Checklist (Based on The Program Evaluation Standards), 1999. 



Page 39 



GAO-11-383 Catastrophic Planning 



Appendix I: Scope and Methodology 



were considered, but discussions with officials in the Grant Programs 
Directorate helped to clarify and narrow the scope of our review to the 
most appropriate candidates for the types of catastrophic planning 
described in the TFER grant. We collected fiscal year 2010 funding levels 
for these grants, as well as information about how funds could be used for 
planning, funding restrictions, and eligibility requirements. 

To identify the extent to which the five states participating in FEMA's 
TFER pilot program addressed the pilot program's stated objectives, we 
analyzed information that FEMA provided in the Pilot Information 
Package and further defined in its TFER Evaluation Criteria document. 
The pilot program's three stated objectives were building relationships, 
developing plans, and documenting lessons learned, and FEMA provided 
several criteria to measure progress towards addressing each of these 
objectives. Additionally, FEMA identified key components to each 
objective. We used FEMA's criteria to assess states' progress toward 
addressing the three program objectives. Moreover, we identified 
additional criteria for the building relationships objective in FEMA's TFER 
Pilot Information Package, including whether pilot states coordinated with 
other states and with FEMA. These components were identified as key 
components in the TFER Pilot Information Package, but were not 
identified in the TFER Evaluation Criteria. Therefore, we included the 
additional criteria in assessing the states' progress toward addressing the 
building relationships objective. We assigned a numerical ranking to 
represent states' progress for each objective's components based on 
interviews we conducted and our review and analysis of documents we 
obtained during our site visits that included draft plans, progress reports, 
and status briefings. We gathered information related to the lessoned 
learned objective from state submissions to a report published by the 
National Guard Bureau in August 2010, and we obtained additional lessons 
learned during our interviews with state officials. 3 Further, two of the 
states provided us with the lessons learned they submitted to FEMA for its 
evaluation, which is not yet complete. 4 If the state met the criterion for a 
particular component, we assigned it a ranking of '5', if the state partially 
met the criterion, we assigned it a ranking of '3', and if the state did not 



3 The National Guard Bureau, Task Force for Emergency Readiness (TFER) Program: 
Report to the National Guard Bureau Strategic Advisory Group Leadership Council 
(Washington, D.C.: August 2010). 

4 FEMA regional offices responded for the remaining three states. 
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meet the criterion, we assigned it a ranking of '0'. Next, we averaged the 
components for each of the three objectives to obtain an overall objective 
score. We then assigned designations to the overall percentages of scores 
as follows: averages greater than 80 percent were designated as 
'substantial progress'; averages from 20 percent to 80 percent were 
designated as 'some progress'; and averages less than 20 percent were 
designated as 'little or no progress.' We did not assign designations to four 
criteria pertaining to developing plans and six criteria pertaining to 
lessons learned because we determined that these criteria do not inform 
the extent to which states met the objective and/or there was not enough 
information to quantify the results. 

States did not receive a designation of "substantial progress" for each 
objective unless they met the criteria related to the key component of that 
objective, that is, (1) developing actual plans — measured by the 
completion of draft plans; and (2) building relationships with FEMA 
officials, Defense Coordinating Officers, and Emergency Preparedness 
Liaison Officers. FEMA did not identify key components of the third 
objective, documenting lessons learned. In instances where states received 
an average above 80 percent but did not meet the criteria related to these 
key components, we assigned a designation of "some progress" for that 
particular objective. For example, by adding the rankings assigned to 
South Carolina for building relationships, the state received a score of 31 
out of 35, or 89 percent. However, South Carolina did not fully meet the 
criteria for coordinating with DOD and FEMA; so the state received the 
designation of 'some progress.' 

Where possible, we used the Comprehensive Preparedness Guide (CPG) 
101 to determine the extent to which the states' plans adhered to FEMA's 
suggested guidelines regarding plan structure and content. About one half 
of the TFER states' 32 planning initiatives involved developing plans. 
Specifically, CPG 101 recommends that basic emergency operations plans 
and their annexes include the following sections: purpose, situation 
overview, and planning assumptions; concept of operations; organization 
and assignment of responsibilities; direction, control, and coordination; 
information collection and dissemination; administration, finance, and 
logistics; and authorities and references. CPG 101 provides additional 
guidance for the structure of the concept of operations section for hazard- 
specific annexes — plans South Carolina developed — and functional 
annexes — one of which Washington developed. We did not evaluate the 
content of these plans because the plans remain in draft form. CPG 101 
does not offer specific content guidance for the other plans types of plans 
TFER states developed. We did not assign a progress designation for the 
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criterion that asked the extent to which states' plans adhered to CPG 101 
because all but 3 of the plans TFER states are developing remain in draft 
form. 

To address these objectives, we interviewed officials and obtained 
information and related documents from federal agencies, the five pilot 
states, and other relevant entities. Within FEMA, we met with officials 
from the National Preparedness Directorate, the Grants Programs 
Directorate, the Response Directorate, and officials in FEMA Regions I, III, 
IV, IX, and X. Within the Department of Defense (DOD), we met with 
representatives from the Office of the Assistant Secretary for Homeland 
Defense and America's Security Affairs and the Joint Chiefs of Staff. We 
also met with U.S. Northern Command and U.S. Pacific Command officials 
because they are the combatant commanders whose areas of 
responsibilities include the United States and its territories. Further, we 
met with the U.S. Department of Homeland Security, the National Guard 
Bureau, the Institute for Defense Analyses, and the National Governor's 
Association. We conducted site visits to all five participant states from 
June through September 2010. During our site visits, we met with the 
planners hired through the TFER pilot program and other state, local, 
National Guard, and DOD regional officials to discuss FEMA's 
management of the TFER pilot program and the states' implementation of 
it. 

We conducted this performance audit from May 2010 to April 2011 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the evidence 
obtained provides a reasonable basis for our findings and conclusions 
based on our audit objectives. 



Page 42 



GAO-11-383 Catastrophic Planning 



Appendix II: Other FEMA Grants Available 
for Catastrophic Planning and States' Views 
on Using These Grants for Planning 



We identified four primary FEMA grant programs that states and localities 
can use for catastrophic planning purposes, as shown in table 2. 



Table 2: FEMA Grant Programs That States and Localities Can Use for Catastrophic Planning (fiscal year 2010 funding levels) 



Eligible recipients 



Basis of fund allocation Funding restrictions 



Use of funds for planning 



State Homeland Security Program ($842 million) 



All 50 states, the District of 
Columbia, Puerto Rico, 
American Samoa, Guam, 
Northern Mariana Islands, 
and the U.S. Virgin Islands 



Allocated based on three 
factors: minimum amounts 
as legislatively mandated, 
DHS's risk methodology, 
and prior year effectiveness. 



States must pass through at least 80 
percent of funds to local 
governments. At least 25 percent of 
total combined funds appropriated for 
the State Homeland Security Program 
and Urban Areas Security Initiative 
must be dedicated towards law 
enforcement terrorism prevention 
activities. Activities implemented 
under this grant must support 
terrorism preparedness. However, 
many capabilities which support 
terrorism preparedness 
simultaneously support preparedness 
for other hazards. Grantees must 
demonstrate this dual-use quality for 
any activities implemented under this 
program that are not explicitly 
focused on terrorism preparedness. 
Grantees may provide an optional 
cost share. 



Funds may be used for planning 
efforts that enable states to 
prioritize needs, build capabilities, 
update preparedness strategies, 
allocate resources, and deliver 
preparedness programs across 
disciplines and levels of 
government. All planning efforts 
should focus on prevention, 
protection, response, and recovery 
efforts, which are the four core 
homeland security mission areas. 
States are encouraged to use 
these funds to leverage other 
specialized grant programs like the 
Regional Catastrophic Planning 
Grant Program. 



Urban Areas Security Initiative ($833 million) 



64 highest risk urban areas 



Allocated based on DHS's 
risk methodology and 
effectiveness - 10 highest 
risk urban areas are 
designated Tier I and the 
remaining 54 are designated 
Tier II urban areas. 



States must pass through at least 80 
percent of funds to local 
governments. At least 25 percent of 
total combined funds appropriated for 
the State Homeland Security Program 
and Urban Areas Security Initiative 
must be dedicated towards law 
enforcement terrorism prevention. 
Activities must support terrorism 
preparedness. However, many 
capabilities which support terrorism 
preparedness simultaneously support 
preparedness for other hazards. 
Grantees must demonstrate the dual- 
use quality for any activities 
implemented that are not explicitly 
focused on terrorism preparedness. 
Any funds retained by the state must 
be used in direct support of the urban 
area. 



Funds may be used for planning 
efforts that enable states to 
prioritize needs, build capabilities, 
update preparedness strategies, 
allocate resources, and deliver 
preparedness programs across 
disciplines and levels of 
government. All planning efforts 
should focus on prevention, 
protection, response, and recovery 
efforts, which are the four core 
homeland security mission areas. 
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Eligible recipients 



Basis of fund allocation Funding restrictions 



Use of funds for planning 



Emergency Management Performance Grants ($329.8 million) 



All 50 states, the District of 
Columbia, Puerto Rico, 
American Samoa, Guam, 
Northern Mariana Islands, 
and the U.S. Virgin Islands, 
as well as the Republic of 
the Marshall Islands and the 
Federated States of 
Micronesia 



Allocated based on a 
designated base amount of 
the total available 
appropriated funds (0.75 
percent for each of the 50 
states, the District of 
Columbia, and the 
Commonwealth of Puerto 
Rico; and 0.25 percent for 
each of the four territories - 
American Samoa, Guam, 
the Northern Mariana 
Islands, and the U.S. Virgin 
Islands). The balance of 
funds will be distributed on a 
population-share basis. 



Required to provide a 50 percent cost 
share or cash or in-kind match. a 



Fiscal year 2010 funds are used 
for a range of emergency 
management planning activities. 
These plans include aspects of the 
following, but are not limited to: (1) 
modifying existing incident 
management and emergency 
operations plans, (2) 
developing/enhancing all-hazards 
mitigation plans, (3) 
developing/enhancing large-scale 
and catastrophic event incident 
plans, and (4) 

developing/enhancing continuity of 
operations and continuity of 
government plans. 



Regional Catastrophic Planning Grant Program ($33.6 million) 



Eleven predesignated urban 
areas 



Allocated based on the risk 
of a catastrophic incident 
occurring in the region and 
the anticipated effectiveness 
of the proposed projects as 
determined through the 
application review process. 



Up to 20 percent of a site's award 
may be retained by the state for 
implementing or integrating the urban 
area's approved project plans 
throughout the state, or with 
neighboring states. 



Funds may be used for planning 
efforts to address catastrophic 
events, including developing 
contingency agreements that 
address logistics and 
prepositioning of commodities. 
Funded efforts must enable the 
prioritization of needs, building of 
capabilities, updating of 
preparedness strategies, allocation 
of resources, and delivery of 
preparedness programs across 
disciplines and levels of 
government. Funds may be used 
to hire staff for program 
management functions, but not for 
operational duties. 



Source: GAO analysis of FEMA grant information. 

"The requirement to allocate 25 percent of funding towards the planning priority in previous 
Emergency Management Performance Grants was removed from the fiscal year 2010 Emergency 
Management Performance Grants guidance. 



Table 3 shows that these FEMA grant programs provide the states and 
other eligible recipients significantly more funds than TFER provided the 
pilot states. For example, in fiscal year 2010 two of the TFER pilot states 
received more than $6 million from the Emergency Management 
Performance Grant program compared to the $350,000 they each received 
for TFER over a 24-month period. According to FEMA officials, the TFER 
grant represents at most 3 percent of the grant money spent on planning 
by the pilot states, as of March 2010, and all TFER activities are allowable 
under other grant programs. 
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Table 3: FEMA Preparedness Grant Programs Funding Allocated to TFER Pilot 
States and Localities within TFER Pilot States 9 


Fiscal year 2010 millions of dollars 


FEMA grant program 


Hawaii 


Massachusetts 


South 
Carolina 


Washington 


West 
Virginia 


State Homeland 
Security Program 


$6.6 


$15.6 


$7.9 


$18.4 


$6.6 


Urban Areas Security 
Initiative 


$4.8 


$18.9 


$0 


$11.1 


$0 


Emergency 
Management 
Performance Grants 


$3.3 


$6.6 


$5.3 


$6.7 


$3.6 


Regional Catastrophic 
Planning Grant 
Program 


$1.7 


$3.6b 


$0 


$1.7 


$3.6 b 



Source: GAO analysis of FEMA grant information. 



"The five pilot states received $350,000 each for TFER over a 24-month period. 

"Funding was allocated to urban areas that span multiple states. For Massachusetts, Boston is the 
principal city in its urban area, and West Virginia has counties included in the National Capitol Region 
urban area. West Virginia is one of six states that receive funds in its area, and, given the small area 
of West Virginia included in the region, a high-ranking state official said that most of the funds are 
spent in areas outside West Virginia. 

TFER has unique attributes from other FEMA grants. The primary 
difference between TFER and these other grant programs is that the TFER 
pilot states are required to use the grant funds exclusively for planning 
efforts. In contrast, states may use the other FEMA grant program funds 
for a full range of activities such as planning, equipment, training, and 
construction and renovation. Like TFER, the Emergency Management 
Performance Grants Program provides funding to states with no 
requirement to pass through funding to local governments. The Regional 
Catastrophic Planning Grant Program provides funds directly to urban 
areas, but beginning in fiscal year 2010, states were allowed to withhold up 
to 20 percent of the funds to implement or integrate the urban areas' 
approved project plans throughout the state, or with neighboring states. 1 
According to the TFER program manger, FEMA provided the TFER pilot 
states with the option of continuing their efforts using Regional 
Catastrophic Planning Grant Program funds; however the four states that 



1 The president's budget for fiscal year 2012 proposes eliminating funding for the Regional 
Catastrophic Planning Grant program. 
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received funds under both programs chose not to pursue this option. 
Similarly, the State Homeland Security Program requires states to pass 
through at least 80 percent of the funding to local governments. Further, 
the Urban Area Security Initiative provides the funding to urban areas, and 
any funds retained by the state must be used to directly support the urban 
area. Consequently, states cannot use most Regional Catastrophic 
Planning Grant, Urban Areas Security Initiative, and State Homeland 
Security Program funds to conduct statewide catastrophic planning. 
However, officials in one state told us they used 2009 Emergency 
Management Performance Grant funds to hire planners to continue 
planning begun under TFER, and officials in two other states told us they 
may use State Homeland Security Program grant funding to continue the 
catastrophic planning efforts begun under TFER. 
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The TFER pilot program was to emphasize integration of planning efforts 
across sectors, jurisdictions, and functional disciplines, as well as 
integration among state, regional, and federal agencies. The Federal 
Emergency Management Agency (FEMA) developed the following 
questions in collaboration with the participating states to evaluate TFER. 
The evaluation questions were structured around FEMA's stated 
objectives for TFER — to develop plans, build relationships, and document 
lessons learned. 

Our assessment of the extent to which each state addressed FEMA's 
stated objectives for the TFER pilot is presented in appendix IV. We did 
not assess four criteria pertaining to developing plans and six criteria 
pertaining to lessons learned because we determined that these criteria do 
not inform the extent to which states met the objective and/or there was 
not enough information to quantify the results. Further, we did not assess 
the administrative elements of the data collection process or 
administrative elements in appendix IV. Those criteria which we did not 
assess are italicized below. 

FEMA's TFER Pilot Program Evaluation 

1. Development of well coordinated and integrated preparedness plans 
through an effective planning process. 

a. Please provide a summary of the primary TFER 
accomplishments. 

b. Describe efforts to coordinate and integrate plans with state and 
federal partners. 

c. Does TFER supported planning reflect the National 
Preparedness Guidelines and the full cycle of planning steps 
outlined in CPG 101? 

d. Is TFER planning responsive to the National Planning Scenarios 
and/or other threats identified through the state's own all-hazards 
risk assessment? 

e. Are the structure, organization, and content of TFER-developed 
plans / annexes / appendices consistent with that recommended 
in CPG 101? 
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/ How are TFER efforts coordinated and integrated with 
resourcing efforts (e.g., training, exercises, grants, etc.)? 

g. Are TFER developed plans consistent with current state 
emergency plans? 

Are they integrated and not duplicative? 

h. Have actual plans been produced? If so, what is their impact on 
state emergency readiness? 

i. In what areas have TFER supported plans been developed (e.g., 
logistics, coordination, etc.)? 

2. Establishment of cross-sector and interagency relationships and 
protocols. 

a. Does TFER supported planning reflect the formation and 
involvement of the sort of integrated and collaborative planning 
teams described in CPG 101? 

b. Have mechanisms been established / augmented and employed 
to ensure cross-sector, multijurisdictional, and interagency 
planning coordination and integration? 

c. Have TFER supported plans been coordinated with local 
jurisdictions? Which ones? State agencies? Which ones? Federal 
partners? Which ones? 

d. Has a blending of TFER civil-military planning expertise been 
fully exploited to assess, strengthen, and advance state 
catastrophic preparedness planning? 

e. How have TFER efforts incorporated the capabilities of the 
private sector as partners in planning activities? 

3. Documentation of lessons learned and recommendations for 
improvement. 

a. In the opinion of state stakeholders, has the TFER Program 
resulted in demonstrable improvement in state planning 
capability? 
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b. To what degree has state-level TFER planning been 
synchronized with and improved the overall quality and 
effectiveness of planning efforts? 

c. How were TFER planners integrated into the state emergency 
management structure and process? 

d. Has the TFER Program resulted in meaningful 
recommendations for the improvement of catastrophic 
preparedness planning doctrine? 

e. Are TFER lessons learned exportable and useful to other states 
and regions to further the evolution and strengthening of their 
own planning programs? 

f Has the pilot identified any additional planning /protocol / 
procedure shortfalls due to integration / synchronization efforts? 

g. Have any exercise After-Action Reports (AARs) or assessment 
documents been developed by the site in the course of the TFER 
Pilot implementation to evaluate the effectiveness of the pilot? 

h. Please provide recommendations for improvements to increase 
the effectiveness of the catastrophic planning programs. 

FEMA also requested copies of plans developed and collected feedback on 
the administrative aspects of the pilot program. 

1. Data collection process. 

a. Please provide copies of completed / draft plans. 

b. Please provide an update of the program plan, to include the 
steps necessary to complete ongoing projects. 

2. Administrative elements. 

a. Describe any challenges faced managing the TFER funds. 

b. Describe the hiring and retention process for your TFER 
planners. Include time lines, training, and other elements as 
appropriate. 
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c. Describe additional administrative successes and challenges 
in managing the pilot. 
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The Federal Emergency Management Agency (FEMA) launched the Task 
Force for Emergency Readiness Pilot Program (TFER) to improve the 
capacity and integration of efforts to plan a response to catastrophic 
disasters. Under the TFER pilot, FEMA was to grant $350,000 to each of 
five participating states — Hawaii, Massachusetts, South Carolina, 
Washington, and West Virginia. FEMA designed TFER to run 18 months, 
beginning on September 1, 2008, and ending on March 31, 2010, but 
extended the pilot program by an additional 6 months to allow the states 
more time to accomplish their goals. The TFER pilot was to emphasize 
horizontal integration of planning efforts across sectors, jurisdictions, and 
functional disciplines, as well as vertical integration among state, regional, 
and federal agencies. For example, regarding vertical integration, states 
were to integrate planning efforts with the Department of Defense (DOD) 
through its Defense Coordinating Officers and Emergency Preparedness 
Liaison Officers. To help develop plans to respond to catastrophic 
disasters, states could hire up to three full-time planners ideally with 
civilian and military planning expertise. FEMA's stated objectives for the 
TFER pilot were to build relationships, develop plans, and document 
lessons learned, and states were to focus their planning on the National 
Planning Scenarios, which represent examples of the gravest dangers 
facing the United States, including terrorist attacks and natural disasters, 
and have been accorded the highest priority for federal planning efforts. 
FEMA articulated these objectives in the TFER Pilot Information Package 
and the TFER Evaluation Criteria document that was ultimately to be used 
to evaluate the TFER pilot. Both of these documents refer to FEMA's 
Comprehensive Preparedness Guide (CPG) 101 as a source states were to 
use to guide their planning process. 

In our assessment of the extent to which each state had addressed FEMA's 
stated objectives for the TFER pilot, we reviewed those documents, used 
the criteria in them that we determined were measurable and appropriate 
to assess the states' progress towards meeting each objective, assigned a 
numerical ranking to each objective's criteria, and then averaged the 
rankings for each objective's criteria. Finally, we assigned designations to 
these averages. When a state addressed more than 80 percent of the 
criteria for an objective this was designated "substantial progress"; from 
20 percent to 80 percent, "some progress"; and below 20 percent, "little or 
no progress." We gathered information related to the lessoned learned 
objective from state submissions to a report published by the National 
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Guard Bureau in August 2010, and we obtained additional lessons learned 
during our interviews with state officials. 1 Further, two of the states 
provided us with the lessons learned they submitted to FEMA for its 
evaluation, which is not yet complete. 2 We did not assign designations to 
four criteria pertaining to developing plans and six criteria pertaining to 
lessons learned because we determined that these criteria do not inform 
the extent to which states met the objective and/or there was not enough 
information to quantify the results. Four of the five pilot states are still 
spending TFER funds, and the states' status in meeting TFER's stated 
objectives may change as catastrophic planning continues. We assessed 
states' progress as of March 2011. 

HclWaii * n ^§ ure ^' we summarize Hawaii's TFER initiatives, note their status, and 

describe the role of the planners working on them. 



1 The National Guard Bureau, Task Force for Emergency Readiness (TFER) Program: 
Report to the National Guard Bureau Strategic Advisory Group Leadership Council 
(Washington, D.C.: August 2010). 

2 FEMA regional offices responded for the remaining three states. 
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Figure 9: Summary and Status of Hawaii's TFER Initiatives and the Role of Planners 
Working on Them, as of March 2011 



TFER initiative 


Status 


Planners' role 


Various logistics and resource management 
plans 

• Include the Resource Management Plan, 
Hawaii Logistics Plan, and Mutual Aid Plans 


Ongoing 


Research and provide 
input to state officials 
writing the plans 


Logistics synchronization matrix 

• Identifies (1) interdependencies among six task 
forces (e.g., those for mass evacuation, energy, 
and debris removal) that would respond to a 
major hurricane making landfall in Honolulu and 
(2) actions taken by those task forces 


Draft products 


Engage all six task 
forces and the Statewide 
Logistics Working Group 
and create the logistics 
synchronization matrix 


Mass evacuation/reception initiative with 
Western states 

• Develops a pilot program with Arizona, Califor- 
nia, and Washington to establish reception sites to 
receive evacuees for further placement/movement 


Ongoing 


Coordinate efforts with 
Arizona, California, and 
Washington 


Disaster management software tool 
(WebEOC) 

• Coordinates responses to catastrophic events 


Ongoing 


Link other state plans 
with this software 



Source: GAO analysis of state information. 



Hawaii is still spending TFER funds and has two TFER planners. As of 
March 2011, Hawaii had made some progress in addressing FEMA's stated 
pilot program objectives. In figure 10, we depict the funding expenditure 
status of the Hawaii TFER pilot and our assessment of the progress made 
in the pilot program towards addressing FEMA's stated pilot program 
objectives. 
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Figure 10: Expenditure Status of the Hawaii TFER Program and Our Assessment of 
Progress Made towards Addressing FEMA's Objectives, as of March 2011 






# MM 



$93,000 


2 planners 


2 planners 


Planning 


L of funding 


originally 


currently 


for hurricane 


* remaining 


hired 


employed 





GAO 

State assessment Status 


Development of 
well-coordinated 
and integrated 
preparedness plans 
through an effective 
planning process 


Q 


State officials reported that TFER planning integrates with the state's 
existing hurricane planning efforts, which relates to a National 
Planning Scenario. State officials told us that TFER planners are 
following the steps outlined in CPG 1 01 , but have not completed the 
full cycle of planning steps for its products, as Hawaii has not 
completed its plans. 


Establishment of 
cross-sector and 
interagency 
relationships and 
protocols 


Q 


Hawaii developed planning teams that included planners from the 
state civil defense division and other state agencies, as well as 
relevant county, federal, and private sector stakeholders. State 
planners told us they had little contact with DOD entities. According to 
state officials, the TFER planner's prior military experience was 
helpful, but not necessary for a civilian planner to be successful. 


Documentation of 
lessons learned and 
recommendations 
for improvement 


Q 


Hawaii officials provided recommendations, including: continue and 
expand the program to other states, maintain flexibility to determine 
work plans, and dedicate funding for catastrophic planning. Addition- 
ally, state officials recommended that future catastrophic planning 
programs have longer time frames than the TFER pilot program, 
provide more clearly defined guidance, and greater coordination 
between FEMA and DOD. Further, funding was not enough to provide 
competitive salaries. Hawaii has not exercised any of its plans to 
determine if they will be effective in the event of a catastrophe. 



9 Substantial progress 

© Some progress 

O Little to no progress 

Source: GAO analysis of state information. 



In figure 11, we detailed figure 10's assessment of the progress made in the 
Hawaii TFER pilot program towards addressing FEMA's stated objectives 
by (1) listing the criteria from the TFER Pilot Information Package and the 
TFER Evaluation Criteria document used to evaluate the progress of each 
objective and (2) providing our assessment score for each criterion. 
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Figure 11: Criteria Used to Assess Progress Made in Hawaii TFER Pilot Program towards Addressing FEMA Objectives, as of 
March 201 V 



Development of well-coordinated and integrated preparedness plans through an effective planning process. © 


Describe efforts to coordinate and integrate 
plans with state and federal partners. 


• 


According to state officials, Hawaii coordinated with state agencies such as the Department of Health 
and Department of Labor and Industrial Relations; and federal agencies such as Transportation Security 
Administration and the General Services Administration. 


Does TFER supported planning reflect the National 
Preparedness Guidelines and the full cycle of planning 
steps outlined in CPG 101? 




TFER planners stated they are following the steps outlined in CPG 1 01 , but have not completed the full 
cycle of planning steps for its products. 


Is TFER planning responsive to the National Planning 
Scenarios and/or other threats identified through the 
state's own all-hazards risk assessment? 


• 


Hawaii based its TFER Pilot Program project areas on the 2006 Nationwide Plan Review Phase 2 
Report, which cited a scenario involving the impact of a major hurricane. 


Are TFER developed plans consistent with current state 
emergency plans? Are they integrated and not 
duplicative? 


• 


The logistics plans integrate with the state's existing hurricane planning efforts, are consistent with state 
plans, and are not duplicative. 


Have actual plans been produced? 


e 


Hawaii has provided us with two draft plans. However, the plans are incomplete and work is ongoing. 




Does TFER supported planning reflect the formation and 
involvement of the sort of integrated and collaborative 
planning teams described in CPG 101? 


• 


Hawaii inserted its TFER planners into its existing planning team in the state's civil defense division. The 
team includes other state agencies, as well as relevant local, federal, and private sector stakeholders. 


Have mechanisms been established/augmented and 
employed to ensure cross-sector, multijurisdictional, and 
interagency planning coordination and integration? 


• 


The TFER planners coordinated with existing interagency taskforces as well as the statewide Logistics 
Working Group. 


Have TFER supported plans been coordinated with local 
jurisdictions? State agencies? Which ones? Federal 
partners? Which ones? 


• 


According to state officials, Hawaii coordinated with local jurisdictions such as county and city officials; 
state agencies such as the Department of Transportation; and federal agencies such as Transportation 
Security Administration and General Services Administration. 


Has a blending of TFER civil-military planning expertise 
been fully exploited to assess, strengthen, and advance 
state catastrophic preparedness planning? 




While Hawaii TFER officials agreed that a blend of civilian-military expertise is helpful and one of the 
TFER planners and the TFER supervisor had prior military experience, they did not feel it was necessary 
in order to be a successful civilian planner. Officials stated that the TFER planners did not have frequent 
contact with the Defense Coordinating Officer or other personnel in the Defense Coordinating Element. 
However, the Defense Coordinating Officer participates in TFER workgroups. Further, they did not 
coordinate with Emergency Preparedness Liaison Officers - part-time military personnel who work 
primarily on the weekends. Hawaii TFER planners coordinated with the National Guard as the Adjutant 
General oversees the National Guard and the state civil defense division where the TFER planners were 
placed, but this coordination was limited by deployments. 


How have TFER efforts incorporated the capabilities of 
the private sector as partners in planning activities? 


• 


Hawaii included Target and Walmart in its planning processes, and state officials told us they included 
nongovernmental organizations such as the Red Cross and Salvation Army in their planning work groups. 


Have TFER pilot states coordinated with other states? 


• 


Hawaii planners coordinated with the TFER pilot state of Washington, along with the non-TFER states 
Oregon, California, Nevada, and Arizona in its hurricane exercise. Hawaii planners visited Florida, North 
Carolina, and Washington to learn more about how these states developed catastrophic planning tools. 


Have TFER pilot states coordinated with FEMA? 




TFER planners told us they coordinated frequently with FEMA's Pacific Area Office within FEMA Region 
IX, but did not coordinate regularly with FEMA Headquarters. Specifically, FEMA provided technical 
assistance and trainings. 
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Documentation of lessons learned and recommendations for improvement, w 


Have any exercise after-action reports or 
assessment documents been developed by the 
site in the course of the TFER Pilot implementa- 
tion to evaluate the effectiveness of the pilot? 


o 


According to state officials, they have not exercised their plans. As such, the state has not developed 
any exercise after-action reports. 


Please provide recommendations for improvement to 
increase the effectiveness of the catastrophic planning 
programs. 


• 


Hawaii officials provided a number of recommendations, including: continue and expand the program to 
other states, maintain flexibility to determine work plans, and dedicate funding for catastrophic planning. 
Additionally, state officials recommended that future catastrophic planning programs have longer time 
frames than the TFER pilot program, provide more clearly defined guidance, and greater coordination 
between FEMA and DOD. Further, funding was not enough to provide competitive salaries. 



• Substantial progress 
© Some progress 
O Little to no progress 

Source: GAO analysis of state information. 

°We selected these criteria from the TFER Evaluation Criteria document and the TFER Pilot 
Information Package. We did not assess 10 of the criteria, as described in app. I, because they do not 
inform the extent to which states met the objective and/or there was not enough information to 
quantify the results. 

We gathered additional information on Hawaii's efforts to develop plans 
and document lessons learned for the criteria we were unable to quantify. 
For example, we analyzed two Hawaii draft plans, which did not contain 
all of the CPG 101 components. However, we were unable to fully assess 
the content of the plans because they are incomplete and work is ongoing. 
Further, Hawaii officials stated that TFER allowed Hawaii to focus on 
logistics planning, and without TFER, Hawaii would not be able to 
undertake this planning. Hawaii reported that TFER has clearly 
contributed to the development of planning task forces and has provided a 
significant contribution towards state catastrophic planning. Additionally, 
Hawaii officials stated that the TFER logistics planning can be used by 
island territories such as Guam and Puerto Rico with similar logistical 
challenges. 
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In figure 12, we summarize Massachusetts's TFER initiatives, note their 
status, and describe the role of the planners working on them. 



Figure 12: Summary and Status of Massachusetts' TFER Initiatives and the Role of Planners Working on Them, as of March 
2011 



TFER initiative 


Status 


Planners' role 


Cape Cod Emergency Traffic Plan 

• Facilitates egress of the high volume of traffic from Cape Cod in 
the event of a hurricane, particularly during the peak tourist 
season, as well as prepares for the temporary sheltering of 
displaced persons 


Updated annually 


Coordinate planning 
efforts for transient 
shelter operations and 
shelter medical services 


Force package planning 

• Provides local jurisdictions Massachusetts National Guard 
domestic response capabilities, should locales need these 
resources during an emergency 

• Clarifies capabilities and capability gaps in state emergency plans 


Draft product 


Draft packages 


Massachusetts state defense force 
revitalization 

• Provides a cadre of skilled professionals capable of assisting the 
state in an emergency 

•Trains medical response force so that out-of-state assistance is 
not necessary 


Cadre is operational and expected to be fully 
staffed by mid-201 1 


Establish the defense 
force and recruited its 
leadership 


Statewide medical support planning 

• Focuses on leveraging federal resources (e.g., Federal Medical 
Stations) while increasing the state's self-reliance, as the State 
Defense Force could operate these stations 


State's Department of Public Health is 
coordinating with the U.S. Department of 
Health and Human Services as well as the 
National Guard 


Facilitate collaborative 
planning among various 
state and federal 
stakeholders to create a 
statewide medical 
support plan 


Input critical infrastructure into database 

• Helps to ensure that state and local first responders had access 
to sufficient information about the Massachusetts National Guard 
infrastructure for their emergency response planning 


TFER planners have added the National 
Guard infrastructure to the Automated Critical 
Asset Management System database and are 
encouraging state agencies and the private 
sector to do the same 


Work with the Common- 
wealth Fusion Center to 
add 24 National Guard 
facilities into the 
database 


Joint reception, staging, onward movement, and integration 

• Establishes written rules and guidelines for the reception, 
tracking, and demobilization of emergency responders following 
or in anticipation of a significant preplanned event, major disaster, 
or emergency 

• Includes FEMA Region I and all states within the region 


Existing planning efforts may be integrated 
into DOD regional planning 


Collect data on the 
state's capabilities to 
support these opera- 
tions 



Source: GAO analysis of state information. 



Massachusetts spent all of its TFER funds and therefore does not have any 
TFER planners employed. As of March 2011, Massachusetts had made 
some progress in meeting FEMA's stated pilot program objectives. In 
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figure 13, we depict the funding expenditure status of the Massachusetts 
TFER pilot and our assessment of the progress made in the pilot program 
towards addressing FEMA's stated pilot program objectives. 



Figure 13: Expenditure Status of the Massachusetts TFER Pilot and Our 
Assessment of Progress Made towards Addressing FEMA's Objectives, as of March 
2011 




GAO 

State assessment Status 


Development of 
well-coordinated 
and integrated 
preparedness plans 
through an effective 
planning process 


Q 


Massachusetts focused on initiatives pertaining to a hurricane and 
other priorities identified by state emergency management officials 
and TFER planners. The balance of its efforts did not specifically 
pertain to National Planning Scenarios. State officials said the 
planners followed the steps outlined in CPG 1 01 , but could not 
complete the full cycle of planning because state officials decided that 
the Massachusetts TFER planners would coordinate planning efforts 
but plans would be completed by other state agencies. 


Establishment of 
cross-sector and 
interagency 
relationships and 
protocols 


w 


Rather than developing plans, TFER planners served as liaisons to 
help formulate planning teams that worked to improve the state's 
catastrophic preparedness. Planners worked within the state National 
Guard. According to state officials, planning teams included planners 
from the state emergency management division and other state 
agencies, as well as relevant local, federal, and private sector 
stakeholders. State officials told us the planners' military planning 
experience gave them credibility in military circles. However, state 
officials told us that DOD entities were minimally involved. 


Documentation of 
lessons learned and 
recommendations 
for improvement 




Massachusetts officials recommended that FEMA continue and 
expand the program to other states, maintain flexibility to determine 
work plans, and dedicate funding for catastrophic planning. Officials 
stated the TFER planners all had some level of military experience, 
which helped them cope with the logistical challenges and have 
credibility with certain audiences. Additionally, officials recommended 
that future catastrophic planning programs have longer time frames 
than the TFER pilot program and provide more clearly defined 
guidance. According to state officials, they have not exercised any 
plans. 



• Substantial progress 

© Some progress 

O Little to no progress 

Source: GAO analysis of state information. 



In figure 14, we detailed figure 13's assessment of the progress made in the 
Massachusetts TFER pilot program towards addressing FEMA's stated 
objectives by (1) listing the criteria from the TFER Pilot Information 
Package and the TFER Evaluation Criteria document used to evaluate the 
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progress of each objective and (2) providing our assessment score for 
each criterion. 



Figure 14: Criteria Used to Assess Progress Made in Massachusetts TFER Pilot towards Addressing FEMA Objectives, as of 
March 2011" 



Development of well-coordinated and integrated preparedness plans through an effective planning process Q 


Describe efforts to coordinate and integrate 
plans with state and federal partners. 


• 


Massachusetts TFER planners told us they coordinated TFER planning initiatives with state agencies 
such as the Massachusetts Emergency Management Agency, the Massachusetts Department of Public 
Health, and the Massachusetts State Police; and federal agencies such as the National Guard, U.S. 
Public Health Service, the Department of Health and Human Services, and the U.S. Coast Guard. 


Does TFER supported planning reflect the National 
Preparedness Guidelines and the full cycle of planning 
steps outlined in CPG 101? 




Planners told us they followed the steps outlined in CPG 101 , but could not complete the full cycle of 
planning because state officials decided that the Massachusetts TFER planners would coordinate the 
planning efforts but plans would be completed by other state agencies. 


Is TFER planning responsive to the National Planning 
Scenarios and/or other threats identified through the 
state's own all-hazards risk assessment? 


© 


Massachusetts focused on initiatives pertaining to a hurricane and other priorities identified by state 
emergency management officials and TFER planners. The balance of its efforts did not specifically 
pertain to National Planning Scenarios. 


Are TFER developed plans consistent with current state 
emergency plans? Are they integrated and not 
duplicative? 


• 


The projects chosen addressed gaps in Massachusetts catastrophic planning and are not duplicative. 


Have actual plans been produced? 


o 


No. However, the TFER planners coordinated efforts that resulted in an update to the Cape Cod 
Emergency Traffic Plan to include transient shelter operations and shelter medical services. 


Establishment of cross-sector and interagency relationships and protocols 


Does TFER supported planning reflect the formation and 
involvement of the sort of integrated and collaborative 
planning teams described in CPG 101? 


• 


The focus of the Massachusetts TFER effort was to serve as liaisons to help formulate planning teams 
that worked to improve the state's catastrophic preparedness, and planners worked within the Massachu- 
setts National Guard. Planning teams included planners from the state emergency management division 
and other state agencies, as well as relevant local, federal, and private sector stakeholders. 


Have mechanisms been established/augmented and 
employed to ensure cross-sector, multijurisdictional, and 
interagency planning coordination and integration? 


• 


According to state officials, the TFER program established and employed workgroups for its various 
initiatives. 


Have TFER supported plans been coordinated with local 
jurisdictions? State agencies? Which ones? Federal 
partners? Which ones? 


• 


Massachusetts TFER planners told us they coordinated TFER planning initiatives with local jurisdictions 
such as Barnstable County and local law enforcement; state agencies such as the Massachusetts 
Emergency Management Agency, the National Guard, and the Massachusetts Department of Public 
Health; and federal agencies such as FEMA Region 1 , the U.S. Public Health Service, the Department of 
Health and Human Services, and the U.S. Coast Guard. 


Has a blending of TFER civil-military planning expertise 
been fully exploited to assess, strengthen, and advance 
state catastrophic preparedness planning? 


e 


State officials told us the planners' military planning experience gave them credibility in military circles, 
and all three planners were placed within the National Guard. However, state officials told us that DOD 
entities such as the Defense Coordinating Officer and Emergency Preparedness Liaison Officer were 
minimally involved. Further, the Defense Coordinating Officer from FEMA Region I told us it would be 
difficult for TFER planners to coordinate with Emergency Preparedness Liaison Officers because they 
work on the weekends, and do not often develop plans. 


How have TFER efforts incorporated the capabilities of 
the private sector as partners in planning activities? 


• 


Massachusetts TFER planners told us they coordinated with the private sector. For example, the planners 
gathered critical infrastructure data from the private sector for one of its projects and coordinated with 
nongovernmental organizations such as the American Red Cross and State of Massachusetts Animal 
Response Team. 


Have TFER pilot states coordinated with other states? 


• 


The Massachusetts TFER planners said they had contact with their counterparts in Hawaii and 
Washington as well as North Carolina, a non-TFER state. 


Have TFER pilot states coordinated with FEMA? 





TFER planners told us they coordinated frequently with FEMA Region I, but had limited contact with 
FEMA headquarters. Specifically, Region I provided information on trainings and responded to information 
requests from TFER planners. 
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Documentation of lessons learned and recommendations for improvement Q 


Have any exercise after-action reports or 
assessment documents been developed by the 
site in the course of the TFER Pilot implementa- 
tion to evaluate the effectiveness of the pilot? 


o 


According to state officials, they have not exercised any plans. As such, the state has not developed 
any exercise after-action reports. 


Please provide recommendations for improvement to 
increase the effectiveness of the catastrophic planning 
programs. 


• 


Massachusetts officials provided a number of recommendations, including: continue and expand the 
program to other states, maintain flexibility to determine work plans, and dedicate funding for catastrophic 
planning. Officials stated the TFER planners all had some level of military experience, which helped them 
cope with the logistical challenges, and have credibility with certain audiences. Additionally, officials 
recommended that future catastrophic planning programs have longer time frames than the TFER pilot 
program and provide more clearly defined guidance. 



• Substantial progress 

© Some progress 

O Little to no progress 

Source: GAO analysis of state information. 

"We selected these criteria from the TFER Evaluation Criteria document and the TFER Pilot 
Information Package. We did not assess 10 of the criteria, as described in app. I, because they do not 
inform the extent to which states met the objective and/or there was not enough information to 
quantify the results. 

We gathered additional information from Massachusetts on developing 
plans and documenting lessons learned, but were unable to quantify this 
information using FEMA's evaluation criteria. For example, Massachusetts 
TFER planners told us they often attended state and regional training and 
workshops, which integrated them with resourcing efforts. However, we 
did not assess progress towards meeting this objective because it is 
unclear how these training and workshops inform plan development. 
Further, Massachusetts officials stated that TFER was successful and 
proved valuable in bringing state, local, and county agencies together with 
nongovernmental organizations under the umbrella of state emergency 
planning. Additionally, TFER officials stated it would have been beneficial 
to have TFER planners in other New England states available to 
coordinate with for events that would likely impact the entire region. 
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Soilth Cctroliricl South Carolina focused its TFER pilot on planning in response to terrorist 

attacks. In figure 15, we summarize South Carolina's TFER initiatives, note 
their status, and describe the role of the planners working on them. 



Figure 15: Summary and Status of South Carolina's TFER Initiatives and the Role of Planners Working on Them as of March 
2011 



TFER initiative 


Status 


Planners' role 


Terrorism Prevention and Response Plan 

• Comprises an appendix to the South Carolina Emergency Operations Plan that 
articulates prevention, protection, response, and recovery activities related to a terrorist 
event 


Draft complete. 
Some of the plan's 
concepts have 
been exercised. 


Update plan to address gaps in 
the response and recovery 
sections and draft prevention and 
protection aspects of the plan 


Catastrophic Incident Response Annex 

• Comprises an appendix to the South Carolina Emergency Operations Plan to provide 
guidance and direction for the State Emergency Response Team as it responds to, and 
recovers from, specific catastrophic incidents 


Complete 


Update plan 


Radiological Dispersal Device Attack Response Plan 

• Outlines an organized, timely, and integrated response from local, state, and federal 
responders to a detonation of a radiological dispersal device at a public park in South Carolina 

• Serves as an attachment to the South Carolina Catastrophic Incident Response Annex 


Draft product 


Draft attachment 


10 Kiloton Uranium Improvised Nuclear Device Response Plan 3 

• Outlines an organized, timely, and integrated response from local, state, and federal 
responders to a detonation of an improvised 10 kiloton nuclear device at a major seaport in 
South Carolina 


Draft product 


Draft attachment 


Explosives Attack-Bombing Using Multiple Improvised Explosive Devices Plan 9 

• Guides the assignment of responsibilities and actions of South Carolina Emergency 
Management Division and its associated State Emergency Response Team, when respond- 
ing to requests from South Carolina counties that may be involved in such an incident 


Draft product 


Draft attachment 


Biological Attack: Animal Disease Response Plan" 

• Outlines an organized, timely, and integrated response from federal, state, and local 
responders to an animal disease outbreak 


Ongoing 


Draft attachment 


Biological Event: Aerosol Anthrax Response Plan" 

• Outlines an organized, timely, and integrated response from federal, state, and local 
responders to an aerosol anthrax attack 


Ongoing 


Draft attachment 


Cyber Attack Response Plan 9 

• Outlines an organized, timely, and integrated response from federal, state, and local 
responders to a cyber attack 


Incomplete 


Draft attachment 


National Planning Scenarios response matrices 

• Document the assets and capabilities of state agencies by emergency support function 
for the National Planning Scenarios 


Incomplete 


• Compile matrices for the 
National Planning Scenarios 

• Focus on nuclear, radiological, 
and improvised explosive 
device scenarios 



Source: GAO analysis of state information. 
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South Carolina is still spending TFER funds and currently has two TFER 
planners employed. As of March 2011, South Carolina had made some 
progress in addressing FEMA's stated pilot program objectives of building 
relationships and documenting lessons learned and had made substantial 
progress in developing plans. In figure 16, we depict the funding 
expenditure status of the South Carolina TFER pilot program and our 
assessment of the progress made in TFER towards addressing FEMA's 
stated pilot program objectives. 
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Figure 16: Expenditure Status of the South Carolina TFER Pilot and Our 
Assessment of Progress Made towards Addressing FEMA's Objectives, as of March 
2011 




State 

Development of 
well-coordinated 
and integrated 
preparedness plans 
through an effective 
planning process 



GAO 
assessment 



Status 

South Carolina focused on the National Planning Scenarios related to 
terrorist attacks. The state has completed a revision of its Catastrophic 
Incident Annex and it has completed drafts of its Terrorism Prevention 
and Response Plan and three other plans related to terrorist attacks, 
which officials told us fill gaps in state planning. Planners reported they 
are following the steps outlined in CPG 101, but have not finished the 
full cycle of planning for any of its products, as South Carolina has not 
exercised them. 



Establishment of 
cross-sector and 
interagency 
relationships and 
protocols 



South Carolina provided a number of recommendations, including: 
continue and expand the program, maintain flexibility to determine work 
plans, and dedicate funding for catastrophic planning. Additionally, 
South Carolina officials recommended that future catastrophic planning 
programs have longer time frames than the TFER pilot program, and 
that FEMA should provide more clearly defined guidance. Officials 
identified coordinating security clearances across DHS and DOD as a 
challenge that should be resolved in similar programs in the future. The 
sharing of lessons learned and techniques/procedures with other 
states would have been helpful in developing catastrophic plans 
through the TFER pilot program. South Carolina also cited good initial 
training opportunities through TFER; however these opportunities did 
not extend throughout the life of the program, and thus were not 
available to all South Carolina TFER staff. 



Documentation of 
lessons learned and 
recommendations 
for improvement 



South Carolina provided a number of recommendations, including: 
continue and expand the program, maintain flexibility to determine 
work plans, and dedicate funding for catastrophic planning. Addition- 
ally, South Carolina officials recommended that future catastrophic 
planning programs have longer time frames than the TFER pilot 
program, and that FEMA should provide more clearly defined 
guidance. Officials identified coordinating security clearances across 
DHS and DOD as a challenge that should be resolved in similar 
programs in the future. The sharing of lessons learned and 
techniques/procedures with other states would have been helpful in 
developing catastrophic plans through the TFER pilot program. South 
Carolina also cited good initial training opportunities through TFER; 
however, these opportunities did not extend throughout the life of the 
program and thus were not available to all South Carolina TFER staff. 
Officials stated that an exercise is scheduled for March 201 1 and that 
after-action reports will subsequently be developed. 



• Substantial progress 

© Some progress 

O Little to no progress 

Source: GAO analysis of state information. 
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In figure 17, we detailed figure 16's assessment of the progress made in 
South Carolina's TFER pilot program towards addressing FEMA's stated 
objectives by (1) listing the criteria from the TFER Pilot Information 
Package and the TFER Evaluation Criteria document used to evaluate the 
progress of each objective and (2) providing our assessment score for 
each criterion. 
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Figure 17: Criteria Used to Assess Progress Made in South Carolina's TFER Pilot towards Addressing FEMA Objectives, as of 
March 201 V 



Development of well-coordinated and integrated preparedness plans through an effective planning process. 


Describe efforts to coordinate and integrate 
plans with state and federal partners. 


• 


According to state officials, South Carolina coordinated TFER plans with state entities, including the 
South Carolina Department of Public Safety, Department of Health and Environmental Control, and 
emergency support function team leaders; and federal agencies, including the Department of Homeland 
Security's Office of Intelligence and Analysis, the U.S. Attorney's Office, the Federal Bureau of 
Investigation, and FEMA Region IV. 


Does TFER supported planning reflect the National 
Preparedness Guidelines and the full cycle of planning 
steps outlined in CPG 101? 




Planners stated they are following the steps outlined in CPG 101 , but have not finished the full cycle of 
planning for any of its products. 


Is TFER planning responsive to the National Planning 
Scenarios and/or other threats identified through the 
state's own all-hazards risk assessment? 


• 


South Carolina focused on the National Planning Scenarios related to terrorist attacks. 


Are TFER developed plans consistent with current state 
emergency plans? Are they integrated and not 
duplicative? 


• 


According to state officials, TFER-developed plans integrate with existing state plans, fill gaps in the 
state plan compendium, are not duplicative, and revise an existing plan to address shortcomings. 


Have actual plans been produced? 


• 


The state is developing response plans for terrorist attacks and is updating its Terrorism 
Prevention and Response Plan. South Carolina has completed an update of its Catastrophic Incident 
Response Annex, has completed drafts of the Terrorism Prevention and Response Plan, and three 
hazard-specific response plans. 


Establishment of cross-sector and interagency relationships and protocols Q 


Does TFER supported planning reflect the formation and 
involvement of the sort of integrated and collaborative 
planning teams? 


• 


South Carolina inserted its TFER planners into the existing planning teams in its law enforcement and 
emergency management divisions. Planning teams also included other state agencies, as well as 
relevant local, federal, and private sector stakeholders. 


Have mechanisms been established/augmented and 
employed to ensure cross-sector, multijurisdictional, and 
interagency planning coordination and integration? 


• 


South Carolina's TFER planners stated they coordinated with established workgroups including leaders 
from the state's emergency support function framework. 


Have TFER supported plans been coordinated with local 
jurisdictions? State agencies? Which ones? Federal 
partners? Which ones? 


• 


According to state officials, South Carolina coordinated TFER plans with local jurisdictions such as local 
law enforcement and emergency management organizations; state entities, including the South Carolina 
Department of Public Safety, Department of Health and Environmental Control, and emergency support 
function team leaders; and federal agencies, including the U.S. Attorney's Office and the Federal Bureau 
of Investigation. 


Has a blending of TFER civil-military planning expertise 
been fully exploited to assess, strengthen, and advance 
state catastrophic preparedness planning? 




TFER and FEMA officials stated the planners' military background helped them learn quickly about 
planning issues and proved to be an asset. A TFER planner in South Carolina told us the state agencies 
involved in TFER coordinated with National Guard officials on a monthly basis and included them in TFER 
workgroups. TFER planners coordinated with the Region's Defense Coordinating Officer and an 
Emergency Preparedness Liaison Officer but the coordination was limited. 


How have TFER efforts incorporated the capabilities of 
the private sector as partners in planning activities? 


• 


Planners consulted with the private sector and included it in workgroups and the statewide exercise. 


Have TFER pilot states coordinated with other states? 


• 


The South Carolina TFER planners told us they regularly exchanged ideas with other states' TFER 
planners. South Carolina shared its Catastrophic Incident Annex with TFER planners in Washington and 
West Virginia. A South Carolina TFER planner also told us she coordinated terrorism planning efforts with 
North Carolina, Georgia, and Tennessee. 


Have TFE pilot states coordinated with FEMA? 


e 


According to state officials, TFER planners coordinated frequently with FEMA Region IV, but had limited 
contact with FEMA headquarters. Specifically, Region IV provided information on technical issues and 
helped the TFER planners build relationships with other stakeholders. 
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Documentation of lessons learned and recommendations for improvement O 


Have any exercise after-action reports or assessment 
documents been developed by the site in the course of 
the TFER Pilot implementation to evaluate the 
effectiveness of the pilot? 


O 


According to state officials, the first of two exercises is scheduled for March 1 0, 201 1 , and after-action 
reports will subsequently be developed. 


Please provide recommendations for improvement to 
increase the effectiveness of the catastrophic planning 
programs. 


• 


South Carolina provided a number of recommendations, including: continue and expand the program, 
maintain flexibility to determine work plans, and dedicate funding for catastrophic planning. Additionally, 
South Carolina officials recommended that future catastrophic planning programs have longer time 
frames than the TFER pilot program, and that FEMA should provide more clearly defined guidance. 
Officials identified coordinating security clearances across DHS and DOD as a challenge that should be 
resolved in similar programs in the future. The sharing of lessons learned and techniques/procedures with 
other states would have been helpful in developing catastrophic plans through the TFER pilot program. 
South Carolina also cited good initial training opportunities through TFER; however, these opportunities 
did not extend throughout the life of the program, and thus were not available to all South Carolina TFER 
staff. The second and third hired South Carolina TFER planners went through an informal in-house 
training procedure. 



• Substantial progress 
© Some progress 
O Little to no progress 

Source: GAO analysis of state information. 

°We selected these criteria from the TFER Evaluation Criteria document and the TFER Pilot 
Information Package. We did not assess 10 of the criteria, as described in app. I, because they do not 
inform the extent to which states met the objective and/or there was not enough information to 
quantify the results. 

We gathered additional information from South Carolina on developing 
plans and documenting lessons learned, but were unable to quantify this 
information using FEMA's evaluation criteria. For example, we were 
unable to fully assess South Carolina's plans because seven of eight TFER- 
developed plans are in draft form. However, all four of the plans state 
officials provided are missing at least one of the recommended sections 
described in CPG 101. State officials told us that omitted sections are 
included in other state planning documents. Additionally, state officials 
reported that the state used TFER to continue work on its existing work 
plan, and South Carolina has developed plans and responses to some of 
the threats that they would not have otherwise been able to address. 
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Washington focused its TFER pilot on developing logistics plans and on a 
catastrophic incident plan. In figure 18, we summarize Washington's TFER 
initiatives, note their status, and describe the role of the planners working 
on them. 



Figure 18: Summary and Status of Washington's TFER Initiatives and the Role of Planners Working on Them, as of March 
2011 



TFER initiative 


Status 


Planners' role 


Catastrophic Incident Annex 

• Establishes the context and the strategy for implementing and coordinating a proactive 
state response to a catastrophic incident. 


Final draft product 


Draft annex from start to finish 


Evacuation Annex 

• Establishes context for a state-managed evacuation 


Draft framework 
completed 


Develop framework from 
beginning 


Reception and Integration Plan Standard Operating Procedures 

• Establishes written rules and guidelines for the reception, tracking, and demobilization of 
emergency responders following or in anticipation of a significant preplanned event, major 
disaster or emergency 

• Supports the Movement Coordination Appendix to the State Comprehensive Emergency 
Management Plan 


Draft product. 
The Emergency 
Management 
Division has 
exercised the plan. 
Final adjustments 
being made. 


Draft product 


Long Term Recovery Appendix 

• Establishes uniform policies for effective coordination to accomplish short-term and 
long-term recovery, restoration, and redevelopment 


Final draft product 


Update appendix 


Movement Coordination Appendix 

•Tracks and coordinates safe, secure, and timely movement of inbound disaster resources 
into the state and through affected jurisdictions following an emergency or disaster 


Draft product 


Draft appendix. 
Participate in the exercises 
pertaining to the procedures 
described in the appendix. 


State Staging Areas Standard Operation Procedures 

• Establishes written rules and guidelines for the selection, activation, operation, and 
demobilization of state staging areas to ensure that the state can receive, track, and 
distribute emergency resources throughout the state in an efficient, effective, and timely 
manner following or in anticipation of a significant preplanned event, major disaster, or 
emergency 


Final product 


Draft procedures. 
Participate in the exercises 
pertaining to these procedures. 


TFER trainings for Local Jurisdictions 

• Prepares local jurisdictions to integrate with state logistics efforts 


Some trainings are 
complete, while 
others are in 
development 


Develop and execute training 
sessions 


Defense Support for Civil Authorities Annex 

• Establishes the circumstances and procedures under which the governor can order the 
Washington National Guard into active duty 

• Coordinates the use of military assets during an emergency or disaster 


Final product 


Update this annex to the State 
Comprehensive Emergency 
Management Plan 



Source: GAO analysis of state information. 



Washington 
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Washington is spending TFER funds and as of March 2011 had one TFER 
planner employed. As of March 2011, Washington had made some progress 
in addressing FEMA's stated pilot program objectives. In figure 19, we 
depict the funding expenditure status of the Washington TFER pilot and 
our assessment of the progress made in the pilot towards addressing 
FEMA's stated pilot objectives. 



Figure 19: Expenditure Status of the Washington TFER Pilot and Our Assessment 
of Progress Made towards Addressing FEMA's Objectives, as of March 2011 
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Development of 
well-coordinated 
and integrated 
preparedness plans 
through an effective 
planning process 


• 


Washington selected as its primary National Planning Scenario a 
catastrophic earthquake and accompanying tsunami. Washington 
integrated TFER logistics plans and a catastrophic incident plan with 
existing state plans and filled gaps in the state plan compendium, 
according to state officials. The state has finalized two of its seven 
plans and has final drafts for three others. Washington officials told us 
they are following CPG 101 planning process and are structuring plan 
content accordingly. 


Establishment of 
cross-sector and 
interagency 
relationships and 
protocols 


Q 


Washington officials stated they developed planning teams that 
included planners from the state Emergency Management Division 
and other state agencies, as well as relevant local, federal, and private 
sector stakeholders. State officials told us the planners' military 
planning experience accelerated the learning of civilian planning 
protocols and helped the program advance the states logistics and 
catastrophic planning. However, according to state officials, DOD 
entities were minimally involved. 


Documentation of 
lessons learned and 
recommendations 
for improvement 


• 


Washington provided recommendations, including: continue and 
expand the program to other states, maintain the broad scope of the 
projects eligible to be addressed through TFER, and dedicate funding 
for catastrophic planning. Additionally, state officials recommended 
that future catastrophic planning programs have longer time frames 
than the TFER pilot program and FEMA provide more guidance. 
Further, FEMA could improve in management of grants like TFER. 
Washington has exercised three out of seven of its TFER plans. 



Substantial progress 
Q Some progress 
O Little to no progress 

Source: GAO analysis of state information. 



In figure 20, we detailed figure 19's assessment of the progress made in 
Washington's TFER pilot program towards addressing FEMA's stated 
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objectives by (1) listing the criteria from the TFER Pilot Information 
Package and the TFER Evaluation Criteria document used to evaluate the 
progress of each objective and (2) providing our assessment score for 
each criterion. 



Figure 20: GAO Assessment of FEMA's Evaluation Criteria: Washington, as of March 2011 " 



Development of well-coordinated and integrated preparedness plans through an effective planning process. 


Describe efforts to coordinate and integrate 
plans with state and federal partners. 


• 


Washington coordinated TFER plans with state agencies such as the Department of Licensing, 
Department of Labor and Industries, Department of General Administration, and the Department of 
Transportation; and federal partners such as FEMA. 


Does TFER supported planning reflect the National 
Preparedness Guidelines and the full cycle of planning 
steps outlined in CPG 101? 


• 


The Washington TFER program supervisor stated the TFER planners are following the steps outlined in 
CPG-1 01 , and have completed the full cycle of planning steps for one of its products. 


Is TFER planning responsive to the National Planning 
Scenarios and/or other threats identified through the 
state's own all-hazards risk assessment? 


• 


Washington selected a catastrophic earthquake (and accompanying tsunami) as the primary scenario for 
TFER, which is one of the national planning scenarios. Specific plans developed for the scenario address 
gaps identified by the state through a gap analysis. 


Are TFER developed plans consistent with current state 
emergency plans? Are they integrated and not 
duplicative? 


• 


TFER-developed plans integrate with existing state plans and fill gaps in the state plan compendium. 
TFER plans are annexes to the state's base emergency operations plan, and are not duplicative. 


Have actual plans been produced? 


• 


The state is developing logistics plans and a catastrophic incident plan for TFER. It has finalized two of 
its seven plans, and has final drafts for three others. 


Establishment of cross-sector and interagency relationships and protocols Q 


Does TFER supported planning reflect the formation and 
involvement of the sort of integrated and collaborative 
planning teams described in CPG 101? 


• 


According to state officials, Washington developed planning teams that included planners from the state 
emergency management division and other state agencies, as well as relevant local, federal, and private 
sector stakeholders. 


Have mechanisms been established/augmented and 
employed to ensure cross-sector, multijurisdictional, and 
interagency planning coordination and integration? 


• 


Washington's TFER program augmented already established practices of coordination and integration 
such as focused workgroups that included representatives from local, state, federal, and nongovernmen- 
tal organizations. 


Have TFER supported plans been coordinated with local 
jurisdictions? State agencies? Which ones? Federal 
partners? Which ones? 


• 


Washington coordinated TFER plans with local jurisdictions including those engaged in the Regional 
Catastrophic Planning Grant; state agencies including the Department of Licensing, Department of Labor 
and Industries, General Administration, and the Department of Transportation; and federal partners 
through FEMA Region X. 


Has a blending of TFER civil-military planning expertise 
been fully exploited to assess, strengthen, and advance 
state catastrophic preparedness planning? 


© 


State officials told us that the planners' military planning experience accelerated the learning of civilian 
planning protocols and helped the program advance the state's logistics and catastrophic planning. 
Washington TFER planners coordinated with the National Guard as the Adjutant General oversees the 
National Guard and the state emergency management division where the TFER planners were placed, 
but this coordination was limited by deployments. However, state officials told us coordination with DOD 
entities such as the Defense Coordinating Officer and Emergency Preparedness Liaison Officer were 
minimally involved. State officials said further coordination with DOD was limited because the draft plans 
were not yet ready for DOD review and coordination with the federal government agencies normally 
occurs through FEMA unless the federal agency is directly involved in plan development. 


How have TFER efforts incorporated the capabilities of 
the private sector as partners in planning activities? 


• 


Private sector groups were involved in the planning and exercising of TFER products. For example, the 
Red Cross participated in one of the TFER workgroups, and multiple private sector associations played 
roles in developing the Reception and Integration Plan and exercise. 


Have TFER pilot states coordinated with other states? 


• 


TFER efforts have been shared with neighboring states and Hawaii to help them understand Washing- 
ton's intentions and processes for logistics response support. 


Have TFER pilot states coordinated with FEMA? 




Washington TFER planners coordinated with FEMA, but officials told us that coordination could be 
improved. Specifically, state officials told us FEMA Region X participated in workgroups and provided 
suggestions on planning, but FEMA headquarters provided limited guidance on the implementation of the 
program, and grant guidance was inconsistent. 
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Documentation of lessons learned and recommendations for improvement 


Have any exercise after-action reports or assessment 
documents been developed by the site in the course 
of the TFER Pilot implementation to evaluate the 
effectiveness of the pilot? 


• 


The state has exercised three of its TFER products during two exercises and developed after-action 
reports for these exercises. 


Please provide recommendations for improvement to 
increase the effectiveness of the catastrophic planning 
programs. 


• 


The state provided a number of recommendations, including: continue and expand the program to other 
states, maintain the broad scope of the projects eligible to be addressed by the TFER resources, and 
dedicate funding for catastrophic planning. Additionally, state officials recommended that future 
catastrophic planning programs have longer time frames than the TFER pilot program and FEMA provide 
more guidance. Further, FEMA could improve in management of grants like TFER. 



• Substantial progress 
O Some progress 
O Little to no progress 



Source: GAO analysis of information. 

"We selected these criteria from the TFER Evaluation Criteria document and the TFER Pilot 
Information Package. We did not assess 10 of the criteria, as described in app. I, because they do not 
inform the extent to which states met the objective and/or there was not enough information to 
quantify the results. 



We gathered additional information from Washington on developing plans 
and documenting lessons learned, but were unable to quantify this 
information using FEMA's evaluation criteria. For example, Washington 
selected projects for TFER which were previously identified as part of the 
state's future emergency planning work. Washington's TFER planning also 
synchronized with catastrophic planning efforts undertaken through the 
Regional Catastrophic Preparedness Grant Program as both efforts 
included evacuation and logistics components. However, we were unable 
to assess the degree to which TFER efforts were synchronized with other 
planning efforts because we did not have a basis for measuring how this 
synchronization improved the effectiveness of planning efforts. Further, 
Washington's TFER plans contain some but not all of the recommended 
sections described in CPG 101, but state and FEMA officials stated that 
omitted sections are included in other state planning documents. We could 
not fully assess the content of Washington's plans because five out of 
seven of the plans are in draft form. According to state officials, 
accomplishments included drafting both catastrophic logistics plans and 
developing a closer working relationship with Hawaii, Oregon, and 
California by participating in Hawaii's annual hurricane exercise. 



WGSt Virginia, West Virginia focused its TFER pilot on developing response plans for 

" three scenarios: a chemical incident, a mass evacuation of the National 

Capital Region, and a dam failure. In figure 21, we summarize West 
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Virginia's TFER initiatives, note their status, and describe the role of the 
planners working on them. 



Figure 21 : Summary and Status of West Virginia's TFER Initiatives and the Role of Planners Working on Them, as of March 
2011 



TFER initiative 


Status 


Planners' role 


Catastrophic Incident Annex 

• Provides procedures that allow emergency manage- 
ment officials to coordinate the application of federal, 
state, and local resources in response to a catastrophic 
event to prevent loss of life, minimize damage to 
property, and protect the environment 


Preliminary draft completed 

Full draft is expected to be completed by summer 201 1 


Draft annex 


Capacity development plans 

• Designs planning tools to help the TFER planners 
assess counties' catastrophic planning strengths and 
weaknesses for each of the three scenarios worked on 
by the planners 


Ongoing - 21 of these planning tools written and at 
varying degrees of completion: 

• 7 for the chemical incident scenario 

• 4 for the National Capitol Region mass evacuation 
scenario 

• 1 for the dam failure scenario 


Draft planning tools 


Narrow the focus tools 

• Identify assets and gaps, such as shelter capacity and 
cots for each of the three scenarios worked on by the 
TFER planners 

• Inform the Capacity Development Plans 


Ongoing 


Develop focus tools 


Bluestone Dam Failure Appendix 

• Informs the federal, state, and local response in the event 
of a failure of the Bluestone Dam 

• Serves as a hazard-specific appendix to the Catastrophic 
Incident Annex 


Ongoing 


Aid plan development 


Kanawha County Evacuation Plan 

• Presents guidelines for emergency services agencies 
in Kanawha County to help ensure a coordinated 
evacuation of the population, including in the event of a 
failure of the Bluestone Dam 


Draft complete 


Ensure consistency between 
the plan and national 
planning doctrine like 
Comprehensive Prepared- 
ness Guide 101 



Source: GAO analysis of state information. 



West Virginia is still spending TFER funds and currently has one TFER 
planner employed. As of March 2011, West Virginia had made some 
progress in meeting FEMA's stated pilot objectives. In figure 22, we depict 
the funding expenditure status of the West Virginia TFER pilot program 
and our assessment of the progress made in the pilot program towards 
addressing FEMA's stated pilot program objectives. 
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Figure 22: Expenditure Status of the West Virginia TFER Pilot and Our Assessment 
of Progress Made towards Addressing FEMA's Objectives, as of March 2011 



WEST VIRGINIA 



$105,000 
of funding 
remaini 
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originally 
hired" 



2 planners Planning for 

currently various terrorist 
employed attacks and hurricane 



GAO 

State assessment Status 


Development of 
well-coordinated 
and integrated 
preparedness plans 
through an effective 
planning process 




West Virginia identified three possible scenarios: a chemical incident, 
a mass evacuation of the National Capital Region, and a dam failure. 
These events could be caused by a hurricane or terrorist attack, which 
pertain to several National Planning Scenarios. According to state 
officials, West Virginia's TFER planning efforts fill gaps in catastrophic 
planning and include development of a catastrophic incident plan. 
State officials said that planners are following the steps outlined in 
CPG 1 01 , but have not finished the full cycle of planning for any of its 
products, as West Virginia has not completed draft plans. 


Establishment of 
cross-sector and 
interagency 
relationships and 
protocols 




West Virginia officials said they developed planning teams that 
included TFER planners, the Department of Military Affairs and Public 
Safety, the National Guard's planning office, other state partners, and 
county partners, as well as the private sector. State officials told us the 
military-civilian blend of planning experience was beneficial in West 
Virginia. However, only one of the two planners has a military 
background and state officials told us coordination with DOD entities 
was minimal. 


Documentation of 
lessons learned and 
recommendations 
for improvement 


Q 


West Virginia recommended FEMA continue and expand the program, 
maintain flexibility to determine workplans and dedicated funding to 
hire planners. Additionally, officials recommended that the new 
program have longer time frames than the pilot and provide more 
clearly defined guidance. West Virginia reported there was significant 
difficulty in hiring people with the requisite military/civilian catastrophic 
planning skills and experience because of the short duration of the 
program, which delayed the start-up of West Virginia's TFER program. 
West Virginia has not exercised any of its plans. 



• Substantial progress 
© Some progress 
O Little to no progress 

Source: GAO analysis of state information. 

"One of the TFER planners is a National Guardsman who was mobilized in September 2010. He is 
expected to resume his TFER planning position when he returns. West Virginia hired another TFER 
planner in December 201 1 . 



In figure 23, we detailed figure 22's assessment of the progress made in 
West Virginia's TFER pilot program towards addressing FEMA's stated 
objectives by (1) listing the criteria from the TFER Pilot Information 
Package and the TFER Evaluation Criteria document used to evaluate the 
progress of each objective and (2) providing our assessment score for 
each criterion. 
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Figure 23: Criteria Used to Assess Progress Made in West Virginia TFER Pilot towards Addressing FEMA Objectives, as of 
March 201 V 



Development of well-coordinated and integrated preparedness plans through an effective planning process. Q 


Describe efforts to coordinate and integrate 
plans with state and federal partners. 


• 


West Virginia officials told us they coordinated TFER planning efforts with the West Virginia National 
Guard, Department of Transportation, Department of Agriculture, and other state agencies, and also 
coordinated with federal partners such as FEMA and the Army Corps of Engineers. 


Does TFER supported planning reflect the National 
Preparedness Guidelines and the full cycle of planning 
steps outlined in CPG 101? 





West Virginia officials stated that the TFER planners are following the steps outlined in CPG 1 01 , but 
have not finished the full cycle of planning for any of its products. 


Is TFER planning responsive to the National Planning 
Scenarios and/or other threats identified through the 
state's own all-hazards risk assessment? 


• 


West Virginia identified three possible scenarios: a chemical incident, a mass evacuation of the National 
Capital Region, and a dam failure. These events could be caused by a hurricane or terrorist attack, which 
pertain to several National Planning Scenarios. 


Are TFER developed plans consistent with current state 
emergency plans? Are they integrated and not 
duplicative? 


• 


West Virginia's TFER planning efforts fill an existing gap in catastrophic planning and are intended to 
integrate with the base emergency operations plan, according to state officials. State officials told us that 
similar planning efforts are underway apart from TFER, but TFER integrates with those efforts as well. 


Have actual plans been produced? 


© 


West Virginia developed a preliminary draft for a catastrophic incident annex and several planning tools 
under TFER. These efforts are ongoing. 


Establishment of cross-sector and interagency relationships and protocols Q 


Does TFER supported planning reflect the formation and 
involvement of the sort of integrated and collaborative 
planning teams described in the Comprehensive 
Preparedness Guide 101? 





West Virginia officials stated that they developed planning teams that included TFER planners, the 
Department of Military Affairs and Public Safety, the National Guard's planning office, other state 
partners, and county partners. However, the state faced challenges coordinating and integrating plans 
with some stakeholders. 


Have mechanisms been established/augmented and 
employed to ensure cross-sector, multijurisdictional, and 
interagency planning coordination and integration? 


• 


According to state officials, the TFER program integrated with already established workgroups such as 
Homeland Security Forums and Local Emergency Planning Committees. 


Have TFER supported plans been coordinated with local 
jurisdictions? State agencies? Which ones? Federal 
partners? Which ones? 


• 


West Virginia officials told us they coordinated TFER planning efforts using a bottom-up approach, 
including coordination at the local level with all 55 counties in the state. Further, TFER planners 
coordinated with the West Virginia National Guard, Department of Transportation, Department of 
Agriculture, and other state agencies, and also coordinated with federal partners such as FEMA and the 
Army Corps of Engineers. West Virginia officials told us that they faced challenges coordinating and 
integrating plans. For example, some state and local stakeholders resisted coordination and overesti- 
mated their planning capacities. 


Has a blending of TFER civil-military planning expertise 
been fully exploited to assess, strengthen, and advance 
state catastrophic preparedness planning? 





State officials told us the military-civilian blend of planning experience was beneficial in West Virginia. 
Specifically, one West Virginia TFER planner is an active member of the state National Guard and 
coordinated regularly with the Guard planning office; the other is a retired fire chief with decades of 
emergency response experience. However, coordination with DOD entities such as the Defense 
Coordinating Officer and Emergency Preparedness Liaison Officer was minimal. State officials said that 
further coordination with DOD was limited because the draft plans were not yet ready for DOD review. 


How have TFER efforts incorporated the capabilities of 
the private sector as partners in planning activities? 


• 


According to state officials, TFER workgroups, such as Local Emergency Planning Committees, include 
multiple private sector stakeholders. 


Have TFER pilot states coordinated with other states? 


• 


State officials told us that, in order to develop a comprehensive Catastrophic Incident Annex, West 
Virginia TFER planners reviewed plans and other planning products from other TFER states such as 
South Carolina and Washington. Specifically, West Virginia is using South Carolina's Catastrophic Incident 
Annex as a template for its own Catastrophic Incident Annex. 


Have TFER pilot states coordinated with FEMA? 





West Virginia officials said planners coordinated infrequently with both FEMA Region III and headquarters 
officials. Regional officials did, however, help them obtain needed information for the dam failure and 
National Capitol Region mass evacuation scenarios. 
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Documentation of lessons learned and recommendations for improvement O 


Have any exercise after-action reports or 
assessment documents been developed by the 
site in the course of the TFER Pilot implementa- 
tion to evaluate the effectiveness of the pilot? 


• 


According to state officials, they have not exercised their plans. As such, the state has not developed 
any exercise after-action reports. 


Please provide recommendations for improvement to 
increase the effectiveness of the catastrophic planning 
programs. 


o 


West Virginia provided a number of recommendations, including: continue and expand the program, 
maintain flexibility to determine workplans and dedicated funding to hire planners for catastrophic 
planning. Additionally, officials recommended that the new program have longer time frames than the 
pilot and provide more clearly defined guidance. West Virginia reported there was significant difficulty in 
hiring people with the requisite military/civilian catastrophic planning skills and experience because of the 
short duration of the program, which delaying the start-up of West Virginia's TFER program. 



• Substantial progress 
© Some progress 
O Little to no progress 

Source: GAO analysis of state information. 



°We selected these criteria from the TFER Evaluation Criteria document and the TFER Pilot 
Information Package. We did not assess 10 of the criteria, as described in app. I, because they do not 
inform the extent to which states met the objective and/or there was not enough information to 
quantify the results. 

We gathered additional information on West Virginia's efforts to develop 
plans and document lessons learned for the criteria we were unable to 
quantify. For example, we analyzed West Virginia's Catastrophic Incident 
Annex, which did not contain all of the CPG 101 components. West 
Virginia reported that the development of this draft annex fulfilled one of 
the main objectives of the pilot program. However, we were unable to fully 
assess the content of the annex because it is incomplete and work is 
ongoing. Additionally, West Virginia TFER planners determined during 
their assessments that there were insufficient personnel available at the 
county level to accomplish the tasks necessary during an incident 
response. In some cases, key individuals were identified as having multiple 
roles and/or responsibilities, which would be impossible to fulfill during an 
actual catastrophic incident. Further, West Virginia reported the "bottom 
up" approach of the West Virginia TFER pilot program is highly effective 
in communicating and working with local jurisdictions, but may prove to 
be less useful in an urban setting. According to state officials, West 
Virginia's TFER efforts are synchronized with the Regional Catastrophic 
Preparedness Grant Program to ensure efforts are not duplicative, 
particularly for the mass evacuation scenario. 
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